On Fri, Apr 11, 2014 at 10:47:14AM +0900, HATAYAMA Daisuke wrote:
From: Vivek Goyal <vgoyal(a)redhat.com>
Subject: Re: [RFC][PATCH] Add --split support for dump on filesystem
Date: Thu, 10 Apr 2014 10:40:03 -0400
> On Wed, Apr 09, 2014 at 03:46:22PM +0900, HATAYAMA Daisuke wrote:
>
> Hi Hatayama,
>
>> So, I'll post a patch to support --split patch as a means of
>> supporting parallelism. Do you agree with this direction?
>
> --split is already an existing parameter which very clearly means
> that split a file into multiple files. I don't know how --split is
> implemented, but I am assuming it creates as many threads as there
> are split files and these threads will do filtering, compression and
> IO.
>
> Usage of --split requires that we *have* to specify multiple files
> as output files.
>
> How about creating a new option say --parallel-dump <nr-threads>. This
> option can take number of threads to create for filetering and compression
> and possibly for doing IO also (to single file).
>
> --parallel-dump will take only single output file as argument. So dump
> is always saved in single file.
>
> We can also make argument <nr-threads> optional. If user specifies number
> of threads, those many threads will be launched. Otherwise makedumpfile
> can detect how many cpus are online and launch those many threads.
>
> For example.
>
> # makedumpfile --parallel-dump 2 /proc/vmcore /var/crash/saved-vmcore
>
> In this case 2 threads will be launched which will coordinate internally
> on filtering and compression and possibly on doing IO too.
>
> # maedumpfile --parallel-dump /proc/vmcore /var/cras/saved-vmcore
>
> In this case makedumpfile will determine how many cpus are online and
> launch as many threads (one thread for each cpu). This is close to "-j"
> option of "make" utility. Difference is that "-j" without option
launches
> as many threads as possible.
>
>
This interface is almost the same as what I think.
So I have a question. How many threads does --split create? Does it
create 1 thread right now to dump to multiple devices?
If yes, we could possibly extend the semantics of --split instead of
introducing new option --parallel-dump.
#makedumpfile --split [<nr-threads>] /proc/vmcore <dump-files>
Now one can specify number of threads to launch to do parallel processing
to save to dump files.
--split should allow even single dump file to be specified.
if nr-threads is not specified, then --split will launch as many threads
as thre are cpus available to take max advantage of parallelism.
#makedupfile --split /proc/vmcore dumpfile1 dumpfile2 ....
- Launch as many threads as there are cpus.
#makedupfile --split N /proc/vmcore dumpfile1 dumpfile2 ....
- Launch N threads.
#makedupfile --split N /proc/vmcore dumpfile
- Launch N threads and save dump to 1 file.
#makedupfile --split /proc/vmcore dumpfile
- Launch as many threads as therea re cpus and save dump to 1 file.
[..]
> Right now we don't seem to have a way to specify destination
vmcore
> file name. If need be, we can possibly create a new option to specify
> file names and then --split option should work.
>
How about the following directive?
# core_collector_vmcore_arguments
#
# - This directive allows you to specify vmcore file
# names to makedumpfile. Default is vmcore.
#
# A specific usecase of this directive is to specify
# multiple vmcore file names for --split option.
# See /sbin/makedumpfile --help for --split option.
#
# This directive is ignored if core_collector is not
# used.
Usage of this directive is like this:
core_collector_vmcore_arguments vmcore-0 vmcore-1 vmcore-2
Users know how many arguments they should specify here from
documentation.
May be something like
vmcore_name vmcore-0 vmcore-1 vmcore-2
Other option could be that we start asking for absolute paths of vmcore
files.
absolute_path /var/crash/vmcore-0 /var/crash/vmcore-1 .....
This way one would automatically decide the underlying disk based on
path and then last string in path will suggest file path.
But I am not sure, I will have to think more about it.
We can first make the case of 1 dump file work and then look into extending
kdump.conf to support multiple files. Above will turn into a more
complicated project. With lots of error checks to make sure all paths lead
to same device. We really don't have a way to support multiple devices
properly.
Thanks
Vivek