On 06/07/2016 05:51 PM, Jaroslav Skarvada wrote:
Hi all,
----- Original Message -----
> Hi Akshay -- curious -- is this mail mainly concerned with ppc64? We have
> profiles to handle latency, though they don't have this knob yet obviously.
>
> There are a few steps:
> 1) get this knob into RHEL
Which knob? Do you mean
/sys/devices/system/cpu/cpufreq/ondemand/sampling_down_factor
?
I can see it in kernel 3.10.0-229.el7.x86_64
I think the proposed change could help with jitter cross architectures, but I have
no figures.
Yes, it's architecture independent as it's related to on-demand/conservative
governor.
But the impact is workload dependent. So doesn't make sense to make it architecture
specific.
> 2) you're free to carry your own tuned profile for your
product
The patch adds balanced-hpc profile. Do we need another profile? Couldn't we just
lower the sampling rate in the original balanced profile (e.g. conditionally if
the knob exists). Would it negatively affect the latency so much to justify new
profile? Does anybody have any figures? Well, in RHEL we mostly use performance or
low latency profiles or derivatives, so it probably shouldn't pose much problem
there.
We could add to the original balanced profile. Ideally, It shouldn't affect the
latency, rather
it should help as it would hold the processors at higher frequency for a longer time.
But as the sampling rate is reduced, the system power would drop slowly, after it reaches
max frequency. So machines running in energy-constrained environment (like battery
powered)
or platforms who care about power more than performance, would not like it. So i feel a
new
profile would be a good idea.
Personally I would rewrite the patch a bit to learn Tuned about the
knob and
not use the script, but this is minor.
I agree, i will rewrite the patch which could be set easily through tuned.conf without the
script.
I was thinking of adding "od_sampling_down_factor" as a config_options in
plugin_cpu.py.
Is this the right place to added? Any other suggestions ?
> 3) we could try to see where this fits in the existing set of
profiles that
> are focus on dejittering and latency and add it there. But not guarantees.
>
> Adding Prarit and Lauro.
>
> This is related to IBM's report here:
>
https://bugzilla.redhat.com/show_bug.cgi?id=1334685
>
> And the request to update cpufreq here:
>
https://bugzilla.redhat.com/show_bug.cgi?id=1342647
>
> On Tue, Jun 7, 2016 at 5:19 AM, Akshay Adiga <
> akshay.adiga(a)linux.vnet.ibm.com> wrote:
>
>> Hi Jaroslav,
>>
>> For HPC workloads its seen that ondemand cpufreq governor introduces some
>> jitter.
>> It has been reported in several distros (eg Ubuntu, Redhat) [1]. Jitter is
>> mainly
>> due to kernel worker threads which needs to get scheduled to perform load
>> calculation and set frequencies.
>>
>> A new cpufreq governor "schedutil" is being developed by Rafael J.
Wysocki
>> which
>> seems to largely solve the problem. It hooks on to the scheduler for load
>> calculations
>> and sets frequency in interrupt contexts (hence no worker thread which
>> causes the problem).
>> But there is a long way to go before it stabilizes and reaches distros.
>>
>> A fix that could reduce jitter went into ubuntu recently (only for ppc64)
>> [1],
>> was by setting a ondemand governor's tuning parameter
>> "sampling_down_factor".
>> Sampling_down_factor is a multiplying factor for sampling rate (rate at
>> which cpu
>> load calculation happens) when the CPU is at its highest frequency it
>> reduces the
>> rate at which load calcuations are done.
>>
>> By setting sampling_down_factor = 100, If cpu reaches max frequency,
>> the sampling rate is reduced by 100 times which reduces jitter.
>>
>> Typically, sampling_rate = 10 ms,
>> sampling_down_factor=100 ,
>> resulting sampling rate = 1000ms = 1 sec
>>
>> The idea is that the effect of jitter is more pronounced when the cpu load
>> is closer to 100%,
>> as ondemand governor sets to max frequency only when there is high cpu
>> load. We could
>> reduce the sampling rate , when it effects the most.
>>
>> Proposed fix through tuned :
>>
>> - Adding a new balanced profile for hpc workloads called balanced-hpc.
>> - We could set sampling_down_parameter to 100 for this profile.
>>
>> [1]
https://bugs.launchpad.net/ubuntu/+source/sysvinit/+bug/1483586
>> [2]
https://www.kernel.org/doc/Documentation/cpu-freq/governors.txt
>>
>> Following is a proposed patch that adds a balanced-hpc profile:
>>
>> This profile derived from the balanced profile. The only changes
>> from balanced profile are the cpufreq governor and additional
>> setting of sampling_down_factor.
>>
>> sampling_down_factor is set to 100.
>>
>> diff --git a/profiles/balanced-hpc/scripts.sh
>> b/profiles/balanced-hpc/scripts.sh
>> new file mode 100755
>> index 0000000..b494e2d
>> --- /dev/null
>> +++ b/profiles/balanced-hpc/scripts.sh
>> @@ -0,0 +1,15 @@
>> +#!/bin/sh
>> +
>> +. /usr/lib/tuned/functions
>> +
>> +start() {
>> + echo 100 >
>> /sys/devices/system/cpu/cpufreq/ondemand/sampling_down_factor
>> + return 0
>> +}
>> +
>> +stop() {
>> + echo 1 >
>> /sys/devices/system/cpu/cpufreq/ondemand/sampling_down_factor
>> + return 0
>> +}
>> +
>> +process $@
>> diff --git a/profiles/balanced-hpc/tuned.conf
>> b/profiles/balanced-hpc/tuned.conf
>> new file mode 100644
>> index 0000000..777ef41
>> --- /dev/null
>> +++ b/profiles/balanced-hpc/tuned.conf
>> @@ -0,0 +1,21 @@
>> +#
>> +# tuned configuration
>> +#
>> +
>> +[cpu]
>> +governor=ondemand
>> +energy_perf_bias=normal
>> +
>> +[audio]
>> +timeout=10
>> +
>> +[video]
>> +radeon_powersave=auto
>> +
>> +[disk]
>> +# Comma separated list of devices, all devices if commented out.
>> +# devices=sda
>> +alpm=medium_power
>> +
>> +[script]
>> +script=scripts.sh
>>
>> Regards
>> Akshay Adiga
>>
>>
>> _______________________________________________
>> power-management mailing list
>> power-management(a)lists.fedoraproject.org
>>
>>
https://lists.fedoraproject.org/admin/lists/power-management@lists.fedora...
>>
>>
>
> --
>
> -- Jeremy Eder
>
thanks & regards
Jaroslav
_______________________________________________
power-management mailing list
power-management(a)lists.fedoraproject.org
https://lists.fedoraproject.org/admin/lists/power-management@lists.fedora...
Regards
Akshay Adiga