Compiler warns about hvt variable in
cpufreq_interactive_speedchange_task() being used without
initialization. Initialize it to ~0ULL.
Note that in reality, this won't happen because when governor_enabled
is true, these two conditions are guaranteed:
1) policy->cpus won't be empty, and
2) target_freq won't be 0.
Otherwise, it indicates more serious issues in the system.
With these two conditions, hvt will be overwritten by at least
one CPU's local_hvttime.
Change-Id: I4378393ed811674f25d54852c296ee5ff407e7e3
Signed-off-by: Junjie Wu <junjiew@codeaurora.org>
This reverts commit ee974af4c9.
sysfs_notify of scaling_cur_freq is no longer used by userspace
components. Since sysfs_notify contends on a single mutex, it could
add long delay to CPU frequency scaling.
Remove the sysfs_notify() to speed up CPU frequency scaling.
Change-Id: I8203904d26bb49e465acd2307574c84f1453bd24
Signed-off-by: Junjie Wu <junjiew@codeaurora.org>
Current implementation of cpufreq_interactive_enable_sched_input()
returns early if use_sched_input is already enabled. This breaks
refcounting for migration notification registration. It could also
result in failure of registering migration notification after
hotplugging the entire cluster and/or suspend/resume.
Change-Id: I079b2c70b182f696cd8a883f5c8e3a37b5c6d21d
Signed-off-by: Junjie Wu <junjiew@codeaurora.org>
down_read_trylock is not always non-blocking if the same thread calls
down_write() before.
CPU1 CPU2
down_read()
down_write()
__down_write_nested()
schedule()
__down_read_trylock()
up_read()
acquires sem->wait_lock
__rwsem_wake_one_writer()
tries to lock sem->wait_lock
Now CPU2 is waiting for CPU1's schedule() to complete, while holding
sem->wait_lock. CPU1 needs sem->wait_lock to continue.
This problem only happens after cpufreq_interactive introduced load
change notification that could be called within schedule().
Add a separate flag to ignore notification if current thread is in
middle of down_write(). This avoids attempting to hold sem->wait_lock.
The additional flag doesn't have any side effects because
down_read_trylock() would have failed anyway.
Change-Id: Iff97cac36c170cf6d03f36de695141289c3d6930
Signed-off-by: Junjie Wu <junjiew@codeaurora.org>
qcom-cpufreq blocks CPU frequency change request during suspend, because
its dependencies might be suspended. Thus a freq change request would
fail silently, and CPU clock won't change until first frequency update
is requested after system comes out of suspend. This creates a period
when thermal driver cannot perform frequency mitigation, even though
policy->min/max have been correctly updated.
Check each online CPU's policy during resume to correct any frequency
violation as soon as possible.
Change-Id: I3be79cf91e7d5e361314020c9806b770823c0b72
Signed-off-by: Junjie Wu <junjiew@codeaurora.org>
Report CPU load to modules subscribed to cpufreq govinfo notification
chain every time governor timer expires to evaluate load.
Change-Id: I0b35947b1924c179649aafa0b7b93d974164af1a
Signed-off-by: Rohit Gupta <rohgup@codeaurora.org>
Some modules can benefit from getting additional information cpufreq
governors use to make frequency switch decisions.
This change lays down a basic framework that the governors can use
to report additional information (Eg: CPU's load) information to
the clients that subscribe to cpufreq govinfo notifier chain.
Change-Id: I511b4bdb7d12394a31ce5352ae47553861e49303
Signed-off-by: Rohit Gupta <rohgup@codeaurora.org>
__cpufreq_get() checks whether policy->cur matches frequency returned
by cpufreq device driver and acts accordingly. However, policy could
be NULL if the CPU is being hotplugged. Current implementation crashes
when trying to dereference the NULL policy.
Return the frequency provided by cpufreq device driver directly if
policy is not available.
Change-Id: I1f2ba4028c259392bf730db37dbec2d8c5ae3fa4
Signed-off-by: Junjie Wu <junjiew@codeaurora.org>
Change 0cd554ae58
(cpufreq: interactive: Exercise hispeed settings at a policy level)
introduced policy level hispeed settings. The change is correct in
general, but it sets hispeed_validate_time after actual frequency
change has completed. Waking up speedchange task and setting
frequency takes non-trival amount of time. This period is not accounted
for in above_hispeed_delay, resulting in additional delays when
ramping up frequency. Frequency switch latency varies a lot
depending on beginning and end frequency, and thus it cannot be easily
compensated by user setting above_hispeed_delay.
Record a local hispeed_validated_time in every CPU's timer function.
Cluster hispeed_validated_time is the local hispeed_validated_time of
CPUs voting for the highest frequency.
Change-Id: Id8ae547fe3a70f8710f60b6e2125954111b7a2b6
Signed-off-by: Junjie Wu <junjiew@codeaurora.org>
cpufreq_register_driver() could race with CPU hotplug during
bootup. Since hotplug notification is not registered when
subsys_interface_register() is being executed, it's possible
cpufreq's view of online CPUs becomes stale before it registers
for hotplug notification.
Register for hotplug notification first and protect
subsys_interface_register() against hotplug using
get/put_online_cpus().
Change-Id: I26b2908f1d167c2becc4e8664c357bb7c6162406
Signed-off-by: Junjie Wu <junjiew@codeaurora.org>
Disable sample window alignment by default to match default behavior
of upstream interactive governor.
Change-Id: Ibbf4bdd4dd423f97d3a9dd5442eba78b378e66e2
Signed-off-by: Junjie Wu <junjiew@codeaurora.org>
When cpufreq_stats_table is not initialized for policy->last_cpu in
CPUFREQ_UPDATE_POLICY_CPU callback, no updates are necessary. Current
implementation dereferences the table unconditionally, causing a crash
if the table is NULL.
Return directly in cpufreq_stats_update_policy_cpu() if
cpufreq_stats_table of policy->last_cpu is NULL.
Change-Id: Ic9ef8120557702791ba5b873532e47cc0a5dc027
Signed-off-by: Junjie Wu <junjiew@codeaurora.org>
Failing to initialize cpufreq_stats table effectively disables
cpufreq_stats. Such failures should not be silent. Print error
messages when cpufreq_stats fails to create stats table.
Change-Id: I71cc0dd8262c7c6946e169f148ae39bd8f213a96
Signed-off-by: Junjie Wu <junjiew@codeaurora.org>
If a heavy task migrates between otherwise idle CPUs in a policy during
every sample window, the above hispeed delay window for the CPUs would get
restarted for every sample window. Due to the continuous restart of above
hispeed delay window, none of the CPUs would ever pick a target frequency
higher than hispeed frequency. This causes the policy's frequency to be
stuck at hispeed freq even if the load justifies a higher frequency.
To fix this, the above high speed delay window is restarted only when the
policy frequency changes. This ensures that tasks migrating between CPUs in
a policy are handled correctly.
Also, the hispeed load/frequency heuristic is only necessary when the
information is insufficient to determine if the load on the CPU needs at
least hispeed frequency. When the policy frequency is already at or above
hispeed frequency, if the CPU load% based on policy frequency is not above
hispeed load, then the information is clearly sufficient to determine that
the load on the CPU does not need hispeed frequency.
Therefore, compute CPU load% (which is used only to compare against hispeed
load) based on policy frequency instead of CPU target frequency.
Change-Id: I1749d663949e34753ecb5c426a16563796f8b0b2
Signed-off-by: Saravana Kannan <skannan@codeaurora.org>
Previously, there was a limitation in load change callback that it
can't attempt to wake up a task. Therefore the best we can do is to
schedule timer at current jiffy. The timer function will only be
executed at next timer tick. This could take up to 10ms.
Now that this limitation is removed, re-evaluate load immediately upon
receiving this callback.
Change-Id: Iab3de4705b9aae96054655b1541e32fb040f7e60
Signed-off-by: Junjie Wu <junjiew@codeaurora.org>
rq->curr/prev_runnable_sum counters represent cpu demand from various
tasks that have run on a cpu. Any task that runs on a cpu will have a
representation in rq->curr_runnable_sum. Their partial_demand value
will be included in rq->curr_runnable_sum. Since partial_demand is
derived from historical load samples for a task, rq->curr_runnable_sum
could represent "inflated/un-realistic" cpu usage. As an example, lets
say that task with partial_demand of 10ms runs for only 1ms on a cpu.
What is included in rq->curr_runnable_sum is 10ms (and not the actual
execution time of 1ms). This leads to cpu busy time being reported on
the upside causing frequency to stay higher than necessary.
This patch fixes cpu busy accounting scheme to strictly represent
actual usage. It also provides for conditional fixup of busy time upon
migration and upon heavy-task wakeup.
CRs-Fixed: 691443
Change-Id: Ic4092627668053934049af4dfef65d9b6b901e6b
Signed-off-by: Srivatsa Vaddagiri <vatsa@codeaurora.org>
Make sampling window alignment optional when scheduler inputs
are not enabled.
Change-Id: If69c111a3efe219cdd1e38c1f46f03404789c0bb
Signed-off-by: Junjie Wu <junjiew@codeaurora.org>
Previously known as sampling down factor, max_freq_hysteresis
extends the period that interactive governor will stay at policy->max.
This feature is to accomodate short idle periods in an otherwise very
intensive workload.
When the feature is enabled, it ensures that once a CPU goes to max
frequency, it doesn't reduce the frequency for max_freq_hysteresis
microseconds from the time it first goes to idle.
Change-Id: Ia54985cb554f63f8c22d0b554a0a0f2ed2be038f
Signed-off-by: Junjie Wu <junjiew@codeaurora.org>
All features in interactive_pro governor is now ported to interactive
governor. Delete interactive_pro governor.
Change-Id: Ic847968f45079ba8d72f1b6fecce4c0e6b88b37a
Signed-off-by: Junjie Wu <junjiew@codeaurora.org>
* commit 'v3.10.49': (529 commits)
Linux 3.10.49
ACPI / battery: Retry to get battery information if failed during probing
x86, ioremap: Speed up check for RAM pages
Score: Modify the Makefile of Score, remove -mlong-calls for compiling
Score: The commit is for compiling successfully.
Score: Implement the function csum_ipv6_magic
score: normalize global variables exported by vmlinux.lds
rtmutex: Plug slow unlock race
rtmutex: Handle deadlock detection smarter
rtmutex: Detect changes in the pi lock chain
rtmutex: Fix deadlock detector for real
ring-buffer: Check if buffer exists before polling
drm/radeon: stop poisoning the GART TLB
drm/radeon: fix typo in golden register setup on evergreen
ext4: disable synchronous transaction batching if max_batch_time==0
ext4: clarify error count warning messages
ext4: fix unjournalled bg descriptor while initializing inode bitmap
dm io: fix a race condition in the wake up code for sync_io
Drivers: hv: vmbus: Fix a bug in the channel callback dispatch code
clk: spear3xx: Use proper control register offset
...
In addition to bringing in upstream commits, this merge also makes minor
changes to mainitain compatibility with upstream:
The definition of list_next_entry in qcrypto.c and ipa_dp.c has been
removed, as upstream has moved the definition to list.h. The implementation
of list_next_entry was identical between the two.
irq.c, for both arm and arm64 architecture, has had its calls to
__irq_set_affinity_locked updated to reflect changes to the API upstream.
Finally, as we have removed the sleep_length member variable of the
tick_sched struct, all changes made by upstream commit ec804bd do not
apply to our tree and have been removed from this merge. Only
kernel/time/tick-sched.c is impacted.
Change-Id: I63b7e0c1354812921c94804e1f3b33d1ad6ee3f1
Signed-off-by: Ian Maund <imaund@codeaurora.org>
Interactive governor does not have enough information about the tasks
on a CPU to make a more informed decision on the frequency the CPUs
should run at. To address this problem, modify interactive governor
to get load information from scheduler. In addition, it can get
notification from scheduler on significant load change to reevaluate
CPU frequency immediately.
Add two sysfs file to control the behavior of load evaluation:
use_sched_load:
When enabled, governor uses load information from scheduler
instead of busy/idle time from past window.
use_migration_notif:
Whenever a task migrates, scheduler might send a notification
so that governor can re-evaluate load and scale frequency.
Governor will ignore this notification unless both
use_sched_hint and use_migration_notification are true for
the policy group.
Change-Id: Iaf66e424c6166ec15480db027002b3a3b357d79c
Signed-off-by: Junjie Wu <junjiew@codeaurora.org>
Replace mod_timer_pinned() with del_timer(), add_timer_on().
mod_timer_pinned() always adds timer onto current CPU. Interactive
governor expects each CPU's timers to be running on the same CPU.
If cpufreq_interactive_timer_resched() is called from another CPU,
the timer will be armed on the wrong CPU.
Replacing mod_timer_pinned() with del_timer() and add_timer_on()
guarantees timers are still run on the right CPU even if another
CPU reschedules the timer. This would provide more flexibility
for future changes.
Change-Id: I3a10be37632afc0ea4e0cc9c86323b9783b216b1
Signed-off-by: Junjie Wu <junjiew@codeaurora.org>
Currently, tunables are only saved to per_cpu field when
CPUFREQ_GOV_POLICY_EXIT event happens. Save tunables the moment they
are created so that per_cpu cached_tunables field always matches
the tunables in use. This is useful for modifying tunable values
across clusters.
Change-Id: I9e30d5e93d6fde1282b5450458d8a605d568a0f5
Signed-off-by: Junjie Wu <junjiew@codeaurora.org>
Timers are scheduled in unit of jiffies. Round up timer_rate so that
it matches the actual sampling period.
Change-Id: I47e666f835752528331f50b1e76784e6d67f8bcf
Signed-off-by: Junjie Wu <junjiew@codeaurora.org>
When CPU has been busy for a long time, last evaluated jiffy will be
quite behind because the timer would have been canceled. We don't want
to schedule a timer to fire in the past as load will always be 100%.
Reset last evaluated jiffy so that timer will be scheduled for the
next window.
Change-Id: Ie25e65eab1f16acdeda267987ca605d653f1f32a
Signed-off-by: Junjie Wu <junjiew@codeaurora.org>
When CPU has been busy for a long time, last evaluated jiffy will be
quite behind because the timer would have been canceled. We don't want
to schedule a timer to fire in the past as load will always be 100%.
Reset last evaluated jiffy so that timer will be scheduled for the
next window.
Change-Id: I4c3838f36bf4d1e4cebce29a26b45611b416d929
Signed-off-by: Junjie Wu <junjiew@codeaurora.org>
It's more advantageous to evaluate all CPUs at same time so that
interactive governor gets a complete picture of the load on
each CPU at a specific time. It could also reduce number of speed
changes made if there are many CPUs controlled by same policy. In
addition, waking up all CPUs at same time would allow the cluster
to go into a deeper sleep state when it's idle.
Change-Id: I6915050c5339ef1af106eb906ebe4b7c618061e2
Signed-off-by: Junjie Wu <junjiew@codeaurora.org>
Interactive governor already has a per_cpu field cpuinfo to keep track
of per_cpu data. Move cached_tunables into cpuinfo.
Change-Id: I77fda0cda76b56ff949456a95f96d129d877aa7b
Signed-off-by: Junjie Wu <junjiew@codeaurora.org>
Remove sampling_down_factor feature.
This commit revert d094d23694
(cpufreq: interactive: Add a sampling_down_factor for max frequencies)
and subsequent modifications related to sampling down factor.
Change-Id: Ib7ec0a918bd3e85a3425dbdeefcd2f2aecffe69c
Signed-off-by: Junjie Wu <junjiew@codeaurora.org>
Sync freq feature is not valid for a HMP system with clusters.
This commit reverts commit f3d1980b4d
(cpufreq: interactive: sync freq feature for interactive governor)
Change-Id: I78cb91a94b1a022f8daed045f5aae69f1c00783d
Signed-off-by: Junjie Wu <junjiew@codeaurora.org>
