3:34 a.m.
Hi,
For HPC workloads its seen that ondemand cpufreq governor introduces some jitter.
It has been reported in several distros (eg Ubuntu, Redhat) [1]. Jitter is mainly
due to kernel worker threads which needs to get scheduled to perform load
calculation and set frequencies.
A new cpufreq governor "schedutil" is being developed by Rafael J. Wysocki
which
seems to largely solve the problem. It hooks on to the scheduler for load calculations
and sets frequency in interrupt contexts (hence no worker thread which causes the
problem).
But there is a long way to go before it stabilizes and reaches distros.
A fix that could reduce jitter went into ubuntu recently (only for ppc64) [1],
was by setting a ondemand governor's tuning parameter
"sampling_down_factor".
Sampling_down_factor is a multiplying factor for sampling rate (rate at which cpu
load calculation happens) when the CPU is at its highest frequency it reduces the
rate at which load calcuations are done.
By setting sampling_down_factor = 100, If cpu reaches max frequency,
the sampling rate is reduced by 100 times which reduces jitter.
Typically, sampling_rate = 10 ms,
sampling_down_factor=100 ,
resulting sampling rate = 1000ms = 1 sec
The idea is that the effect of jitter is more pronounced when the cpu load is closer to
100%,
as ondemand governor sets to max frequency only when there is high cpu load. We could
reduce the sampling rate , when it effects the most.
Proposed fix through tuned :
- Adding a new balanced profile for hpc workloads called balanced-hpc.
- We could set sampling_down_parameter to 100 for this profile.
[1] https://bugs.launchpad.net/ubuntu/+source/sysvinit/+bug/1483586
[2] https://www.kernel.org/doc/Documentation/cpu-freq/governors.txt
Following is a proposed patch that adds a balanced-hpc profile:
This profile derived from the balanced profile. The only changes
from balanced profile are the cpufreq governor and additional
setting of sampling_down_factor.
sampling_down_factor is set to 100.
iff --git a/profiles/balanced-hpc/scripts.sh b/profiles/balanced-hpc/scripts.sh
new file mode 100755
index 0000000..b494e2d
--- /dev/null
+++ b/profiles/balanced-hpc/scripts.sh
@@ -0,0 +1,15 @@
+#!/bin/sh
+
+. /usr/lib/tuned/functions
+
+start() {
+ echo 100 > /sys/devices/system/cpu/cpufreq/ondemand/sampling_down_factor
+ return 0
+}
+
+stop() {
+ echo 1 > /sys/devices/system/cpu/cpufreq/ondemand/sampling_down_factor
+ return 0
+}
+
+process $@
diff --git a/profiles/balanced-hpc/tuned.conf b/profiles/balanced-hpc/tuned.conf
new file mode 100644
index 0000000..777ef41
--- /dev/null
+++ b/profiles/balanced-hpc/tuned.conf
@@ -0,0 +1,21 @@
+#
+# tuned configuration
+#
+
+[cpu]
+governor=ondemand
+energy_perf_bias=normal
+
+[audio]
+timeout=10
+
+[video]
+radeon_powersave=auto
+
+[disk]
+# Comma separated list of devices, all devices if commented out.
+# devices=sda
+alpm=medium_power
+
+[script]
+script=scripts.sh
Regards
Akshay Adiga