Piteball
/
android_kernel_samsung_msm8226
mirror of https://github.com/S3NEO/android_kernel_samsung_msm8226.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

sched_clock: Backport high-resolution support 

sched_clock: Use seqcount instead of rolling our own

We're going to increase the cyc value to 64 bits in the near
future. Doing that is going to break the custom seqcount
implementation in the sched_clock code because 64 bit numbers
aren't guaranteed to be atomic. Replace the cyc_copy with a
seqcount to avoid this problem.

Cc: Russell King <linux@arm.linux.org.uk>
Acked-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Stephen Boyd <sboyd@codeaurora.org>
Signed-off-by: John Stultz <john.stultz@linaro.org>
Git-commit: 85c3d2dd15be4d577a37ffb8bbbd019fc8e3280a
Git-repo: git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
[imaund@codeaurora.org: resolve merge conflicts]
Signed-off-by: Ian Maund <imaund@codeaurora.org>

Change-Id: Ic30515ef65e5d3ec008f03ae020a9599e7c01fb2

sched_clock: Use an hrtimer instead of timer

In the next patch we're going to increase the number of bits that
the generic sched_clock can handle to be greater than 32. With
more than 32 bits the wraparound time can be larger than what can
fit into the units that msecs_to_jiffies takes (unsigned int).
Luckily, the wraparound is initially calculated in nanoseconds
which we can easily use with hrtimers, so switch to using an
hrtimer.

Change-Id: Id6059fae75863ddd3972f4dcac1cf7b803b09ac9
Cc: Russell King <linux@arm.linux.org.uk>
Signed-off-by: Stephen Boyd <sboyd@codeaurora.org>
[jstultz: Fixup hrtimer intitialization order issue]
Signed-off-by: John Stultz <john.stultz@linaro.org>
Git-commit: a08ca5d1089da03724f96fa0870c64968e66765b
Git-repo: git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
Signed-off-by: Ian Maund <imaund@codeaurora.org>

sched_clock: Add support for >32 bit sched_clock

The ARM architected system counter has at least 56 usable bits.
Add support for counters with more than 32 bits to the generic
sched_clock implementation so we can increase the time between
wakeups due to dealing with wrap-around on these devices while
benefiting from the irqtime accounting and suspend/resume
handling that the generic sched_clock code already has. On my
system using 56 bits over 32 bits changes the wraparound time
from a few minutes to an hour. For faster running counters (GHz
range) this is even more important because we may not be able to
execute the timer in time to deal with the wraparound if only 32
bits are used.

We choose a maxsec value of 3600 seconds because we assume no
system will go idle for more than an hour. In the future we may
need to increase this value.

Note: All users should switch over to the 64-bit read function so
we can remove setup_sched_clock() in favor of sched_clock_register().

Change-Id: I81a4b7102db5316bd31d1fa54e2d801f1ee133c1
Cc: Russell King <linux@arm.linux.org.uk>
Signed-off-by: Stephen Boyd <sboyd@codeaurora.org>
Signed-off-by: John Stultz <john.stultz@linaro.org>
Git-commit: e7e3ff1bfe9c42ee31172e9afdc0383a9e595e29
Git-repo: git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
Signed-off-by: Ian Maund <imaund@codeaurora.org>

sched_clock: Avoid corrupting hrtimer tree during suspend

During suspend we call sched_clock_poll() to update the epoch and
accumulated time and reprogram the sched_clock_timer to fire
before the next wrap-around time. Unfortunately,
sched_clock_poll() doesn't restart the timer, instead it relies
on the hrtimer layer to do that and during suspend we aren't
calling that function from the hrtimer layer. Instead, we're
reprogramming the expires time while the hrtimer is enqueued,
which can cause the hrtimer tree to be corrupted. Furthermore, we
restart the timer during suspend but we update the epoch during
resume which seems counter-intuitive.

Let's fix this by saving the accumulated state and canceling the
timer during suspend. On resume we can update the epoch and
restart the timer similar to what we would do if we were starting
the clock for the first time.

Change-Id: Iee2a1cca42e5b681347ea0607e9af420a63892d7
CRs-Fixed: 696826
Fixes: a08ca5d1089d "sched_clock: Use an hrtimer instead of timer"
Signed-off-by: Stephen Boyd <sboyd@codeaurora.org>

sched_clock: Make ARM's sched_clock generic for all architectures

Nothing about the sched_clock implementation in the ARM port is
specific to the architecture. Generalize the code so that other
architectures can use it by selecting GENERIC_SCHED_CLOCK.

Change-Id: I1e846fd1fb87ec1217ea84f1e2e02596c9eaa96b
Signed-off-by: Stephen Boyd <sboyd@codeaurora.org>
[jstultz: Merge minor collisions with other patches in my tree]
Signed-off-by: John Stultz <john.stultz@linaro.org>

Change-Id: Id4ff982d3b5dfdc2fa172f3bdbddc6021134c932
Signed-off-by: Kevin F. Haggerty <haggertk@lineageos.org>
This commit is contained in:
Stephen Boyd 2014-01-21 16:56:38 -08:00 committed by Francescodario Cuzzocrea
parent 098d204821
commit d11ca916b8
25 changed files with 264 additions and 26 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
arch/arm/Kconfig
View File

@ -10,6 +10,7 @@ config ARM
select SYS_SUPPORTS_APM_EMULATION
select GENERIC_ATOMIC64 if (CPU_V6 || !CPU_32v6K || !AEABI)
select HAVE_OPROFILE if (HAVE_PERF_EVENTS)
select GENERIC_SCHED_CLOCK
select HAVE_ARCH_JUMP_LABEL if !XIP_KERNEL
select HAVE_ARCH_KGDB
select HAVE_KPROBES if !XIP_KERNEL

2
arch/arm/common/timer-sp.c
View File

@ -25,8 +25,8 @@
#include <linux/interrupt.h>
#include <linux/irq.h>
#include <linux/io.h>
#include <linux/sched_clock.h>
#include <asm/sched_clock.h>
#include <asm/hardware/arm_timer.h>
static long __init sp804_get_clock_rate(const char *name)

2
arch/arm/kernel/Makefile
View File

@ -16,7 +16,7 @@ CFLAGS_REMOVE_return_address.o = -pg
# Object file lists.
obj-y := elf.o entry-armv.o entry-common.o irq.o opcodes.o \
process.o ptrace.o return_address.o sched_clock.o \
process.o ptrace.o return_address.o \
setup.o signal.o stacktrace.o sys_arm.o time.o traps.o
obj-$(CONFIG_DEPRECATED_PARAM_STRUCT) += compat.o

6
arch/arm/kernel/arch_timer.c
View File

@ -25,12 +25,12 @@
#include <linux/irq.h>
#include <linux/export.h>
#include <linux/slab.h>
#include <linux/sched_clock.h>
#include <asm/cputype.h>
#include <asm/delay.h>
#include <asm/localtimer.h>
#include <asm/arch_timer.h>
#include <asm/sched_clock.h>
#include <asm/hardware/gic.h>
#include <asm/system_info.h>
@ -377,7 +377,7 @@ static u32 arch_counter_get_cntvct32(void)
return (u32)(cntvct & (u32)~0);
}
static u32 notrace arch_timer_update_sched_clock(void)
static u64 notrace arch_timer_update_sched_clock(void)
{
return arch_counter_get_cntvct32();
}
@ -403,7 +403,7 @@ static void __init arch_timer_counter_init(void)
{
clocksource_register_hz(&clocksource_counter, arch_timer_rate);
setup_sched_clock(arch_timer_update_sched_clock, 32, arch_timer_rate);
sched_clock_register(arch_timer_update_sched_clock, 32, arch_timer_rate);
/* Use the architected timer for the delay loop. */
arch_delay_timer.read_current_timer = &arch_timer_read_current_timer;

3
arch/arm/kernel/time.c
View File

@ -24,10 +24,10 @@
#include <linux/syscore_ops.h>
#include <linux/timer.h>
#include <linux/irq.h>
#include <linux/sched_clock.h>
#include <asm/leds.h>
#include <asm/thread_info.h>
#include <asm/sched_clock.h>
#include <asm/stacktrace.h>
#include <asm/mach/arch.h>
#include <asm/mach/time.h>
@ -147,6 +147,5 @@ void __init time_init(void)
{
system_timer = machine_desc->timer;
system_timer->init();
sched_clock_postinit();
}

2
arch/arm/mach-davinci/time.c
View File

@ -18,8 +18,8 @@
#include <linux/clk.h>
#include <linux/err.h>
#include <linux/platform_device.h>
#include <linux/sched_clock.h>
#include <asm/sched_clock.h>
#include <asm/mach/irq.h>
#include <asm/mach/time.h>

1
arch/arm/mach-integrator/integrator_ap.c
View File

@ -41,7 +41,6 @@
#include <asm/setup.h>
#include <asm/param.h> /* HZ */
#include <asm/mach-types.h>
#include <asm/sched_clock.h>
#include <mach/lm.h>
#include <mach/irqs.h>

2
arch/arm/mach-ixp4xx/common.c
View File

@ -29,6 +29,7 @@
#include <linux/io.h>
#include <linux/export.h>
#include <linux/gpio.h>
#include <linux/sched_clock.h>
#include <mach/udc.h>
#include <mach/hardware.h>
@ -37,7 +38,6 @@
#include <asm/pgtable.h>
#include <asm/page.h>
#include <asm/irq.h>
#include <asm/sched_clock.h>
#include <asm/system_misc.h>
#include <asm/mach/map.h>

2
arch/arm/mach-mmp/time.c
View File

@ -25,8 +25,8 @@
#include <linux/io.h>
#include <linux/irq.h>
#include <linux/sched_clock.h>
#include <asm/sched_clock.h>
#include <mach/addr-map.h>
#include <mach/regs-timers.h>
#include <mach/regs-apbc.h>

2
arch/arm/mach-msm/timer.c
View File

@ -24,11 +24,11 @@
#include <linux/io.h>
#include <linux/percpu.h>
#include <linux/mm.h>
#include <linux/sched_clock.h>
#include <asm/localtimer.h>
#include <asm/mach/time.h>
#include <asm/hardware/gic.h>
#include <asm/sched_clock.h>
#include <asm/smp_plat.h>
#include <asm/user_accessible_timer.h>
#include <mach/msm_iomap.h>

2
arch/arm/mach-omap1/time.c
View File

@ -43,10 +43,10 @@
#include <linux/clocksource.h>
#include <linux/clockchips.h>
#include <linux/io.h>
#include <linux/sched_clock.h>
#include <asm/leds.h>
#include <asm/irq.h>
#include <asm/sched_clock.h>
#include <mach/hardware.h>
#include <asm/mach/irq.h>

3
arch/arm/mach-omap2/timer.c
View File

@ -36,11 +36,12 @@
#include <linux/clocksource.h>
#include <linux/clockchips.h>
#include <linux/slab.h>
#include <linux/sched_clock.h>
#include <asm/mach/time.h>
#include <plat/dmtimer.h>
#include <asm/smp_twd.h>
#include <asm/sched_clock.h>
#include "common.h"
#include <plat/omap_hwmod.h>
#include <plat/omap_device.h>

2
arch/arm/mach-pxa/time.c
View File

@ -16,11 +16,11 @@
#include <linux/init.h>
#include <linux/interrupt.h>
#include <linux/clockchips.h>
#include <linux/sched_clock.h>
#include <asm/div64.h>
#include <asm/mach/irq.h>
#include <asm/mach/time.h>
#include <asm/sched_clock.h>
#include <mach/regs-ost.h>
#include <mach/irqs.h>

2
arch/arm/mach-sa1100/time.c
View File

@ -14,9 +14,9 @@
#include <linux/irq.h>
#include <linux/timex.h>
#include <linux/clockchips.h>
#include <linux/sched_clock.h>
#include <asm/mach/time.h>
#include <asm/sched_clock.h>
#include <mach/hardware.h>
#include <mach/irqs.h>

2
arch/arm/mach-u300/timer.c
View File

@ -17,11 +17,11 @@
#include <linux/io.h>
#include <linux/clk.h>
#include <linux/err.h>
#include <linux/sched_clock.h>
#include <mach/hardware.h>
/* Generic stuff */
#include <asm/sched_clock.h>
#include <asm/mach/map.h>
#include <asm/mach/time.h>
#include <asm/mach/irq.h>

2
arch/arm/plat-iop/time.c
View File

@ -22,9 +22,9 @@
#include <linux/clocksource.h>
#include <linux/clockchips.h>
#include <linux/export.h>
#include <linux/sched_clock.h>
#include <mach/hardware.h>
#include <asm/irq.h>
#include <asm/sched_clock.h>
#include <asm/uaccess.h>
#include <asm/mach/irq.h>
#include <asm/mach/time.h>

3
arch/arm/plat-omap/counter_32k.c
View File

@ -18,8 +18,9 @@
#include <linux/err.h>
#include <linux/io.h>
#include <linux/clocksource.h>
#include <linux/sched_clock.h>
#include <asm/sched_clock.h>
#include <asm/mach/time.h>
#include <plat/hardware.h>
#include <plat/common.h>

2
arch/arm/plat-orion/time.c
View File

@ -16,7 +16,7 @@
#include <linux/clockchips.h>
#include <linux/interrupt.h>
#include <linux/irq.h>
#include <asm/sched_clock.h>
#include <linux/sched_clock.h>
/*
* MBus bridge block registers.

2
arch/arm/plat-versatile/sched-clock.c
View File

@ -20,8 +20,8 @@
*/
#include <linux/kernel.h>
#include <linux/io.h>
#include <linux/sched_clock.h>
#include <asm/sched_clock.h>
#include <plat/sched_clock.h>
static void __iomem *ctr;

3
drivers/clocksource/clksrc-dbx500-prcmu.c
View File

@ -14,8 +14,7 @@
*/
#include <linux/clockchips.h>
#include <linux/clksrc-dbx500-prcmu.h>
#include <asm/sched_clock.h>
#include <linux/sched_clock.h>
#include <mach/setup.h>
#include <mach/hardware.h>

13
arch/arm/include/asm/sched_clock.h → include/linux/sched_clock.h
View File

@ -5,10 +5,17 @@
* it under the terms of the GNU General Public License version 2 as
* published by the Free Software Foundation.
*/
#ifndef ASM_SCHED_CLOCK
#define ASM_SCHED_CLOCK
#ifndef LINUX_SCHED_CLOCK
#define LINUX_SCHED_CLOCK
#ifdef CONFIG_GENERIC_SCHED_CLOCK
extern void sched_clock_postinit(void);
extern void setup_sched_clock(u32 (*read)(void), int bits, unsigned long rate);
#else
static inline void sched_clock_postinit(void) { }
#endif
extern void sched_clock_register(u64 (*read)(void), int bits,
unsigned long rate);
#endif

3
init/Kconfig
View File

@ -582,6 +582,9 @@ config LOG_BUF_SHIFT
config HAVE_UNSTABLE_SCHED_CLOCK
bool
config GENERIC_SCHED_CLOCK
bool
menuconfig CGROUPS
boolean "Control Group support"
depends on EVENTFD

2
init/main.c
View File

@ -72,6 +72,7 @@
#include <linux/memblock.h>
#endif
#include <linux/random.h>
#include <linux/sched_clock.h>
#include <asm/io.h>
#include <asm/bugs.h>
@ -708,6 +709,7 @@ asmlinkage void __init start_kernel(void)
softirq_init();
timekeeping_init();
time_init();
sched_clock_postinit();
profile_init();
call_function_init();
if (!irqs_disabled())

1
kernel/time/Makefile
View File

@ -4,6 +4,7 @@ obj-y += timeconv.o posix-clock.o #alarmtimer.o
obj-$(CONFIG_GENERIC_CLOCKEVENTS_BUILD) += clockevents.o
obj-$(CONFIG_GENERIC_CLOCKEVENTS) += tick-common.o
obj-$(CONFIG_GENERIC_CLOCKEVENTS_BROADCAST) += tick-broadcast.o
obj-$(CONFIG_GENERIC_SCHED_CLOCK) += sched_clock.o
obj-$(CONFIG_TICK_ONESHOT) += tick-oneshot.o
obj-$(CONFIG_TICK_ONESHOT) += tick-sched.o
obj-$(CONFIG_TIMER_STATS) += timer_stats.o

225
kernel/time/sched_clock.c Normal file
View File

@ -0,0 +1,225 @@
/*
* sched_clock.c: support for extending counters to full 64-bit ns counter
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License version 2 as
* published by the Free Software Foundation.
*/
#include <linux/clocksource.h>
#include <linux/init.h>
#include <linux/jiffies.h>
#include <linux/ktime.h>
#include <linux/kernel.h>
#include <linux/moduleparam.h>
#include <linux/sched.h>
#include <linux/syscore_ops.h>
#include <linux/hrtimer.h>
#include <linux/sched_clock.h>
#include <linux/seqlock.h>
#include <linux/bitops.h>
struct clock_data {
ktime_t wrap_kt;
u64 epoch_ns;
u64 epoch_cyc;
seqcount_t seq;
unsigned long rate;
u32 mult;
u32 shift;
bool suspended;
};
static struct hrtimer sched_clock_timer;
static int irqtime = -1;
static int initialized;
core_param(irqtime, irqtime, int, 0400);
static struct clock_data cd = {
.mult = NSEC_PER_SEC / HZ,
};
static u64 __read_mostly sched_clock_mask;
static u64 notrace jiffy_sched_clock_read(void)
{
/*
* We don't need to use get_jiffies_64 on 32-bit arches here
* because we register with BITS_PER_LONG
*/
return (u64)(jiffies - INITIAL_JIFFIES);
}
static u64 __read_mostly (*read_sched_clock)(void) = jiffy_sched_clock_read;
static inline u64 notrace cyc_to_ns(u64 cyc, u32 mult, u32 shift)
{
return (cyc * mult) >> shift;
}
unsigned long long notrace sched_clock(void)
{
u64 epoch_ns;
u64 epoch_cyc;
u64 cyc;
unsigned long seq;
if (cd.suspended)
return cd.epoch_ns;
do {
seq = read_seqcount_begin(&cd.seq);
epoch_cyc = cd.epoch_cyc;
epoch_ns = cd.epoch_ns;
} while (read_seqcount_retry(&cd.seq, seq));
cyc = read_sched_clock();
cyc = (cyc - epoch_cyc) & sched_clock_mask;
return epoch_ns + cyc_to_ns(cyc, cd.mult, cd.shift);
}
/*
* Atomically update the sched_clock epoch.
*/
static void notrace update_sched_clock(void)
{
unsigned long flags;
u64 cyc;
u64 ns;
cyc = read_sched_clock();
ns = cd.epoch_ns +
cyc_to_ns((cyc - cd.epoch_cyc) & sched_clock_mask,
cd.mult, cd.shift);
raw_local_irq_save(flags);
write_seqcount_begin(&cd.seq);
cd.epoch_ns = ns;
cd.epoch_cyc = cyc;
write_seqcount_end(&cd.seq);
raw_local_irq_restore(flags);
}
static enum hrtimer_restart sched_clock_poll(struct hrtimer *hrt)
{
update_sched_clock();
hrtimer_forward_now(hrt, cd.wrap_kt);
return HRTIMER_RESTART;
}
void __init sched_clock_register(u64 (*read)(void), int bits,
unsigned long rate)
{
u64 res, wrap, new_mask, new_epoch, cyc, ns;
u32 new_mult, new_shift;
ktime_t new_wrap_kt;
unsigned long r;
char r_unit;
if (cd.rate > rate)
return;
WARN_ON(!irqs_disabled());
/* calculate the mult/shift to convert counter ticks to ns. */
clocks_calc_mult_shift(&new_mult, &new_shift, rate, NSEC_PER_SEC, 3600);
new_mask = CLOCKSOURCE_MASK(bits);
/* calculate how many ns until we wrap */
wrap = clocks_calc_max_nsecs(new_mult, new_shift, 0, new_mask);
new_wrap_kt = ns_to_ktime(wrap - (wrap >> 3));
/* update epoch for new counter and update epoch_ns from old counter*/
new_epoch = read();
cyc = read_sched_clock();
ns = cd.epoch_ns + cyc_to_ns((cyc - cd.epoch_cyc) & sched_clock_mask,
cd.mult, cd.shift);
write_seqcount_begin(&cd.seq);
read_sched_clock = read;
sched_clock_mask = new_mask;
cd.rate = rate;
cd.wrap_kt = new_wrap_kt;
cd.mult = new_mult;
cd.shift = new_shift;
cd.epoch_cyc = new_epoch;
cd.epoch_ns = ns;
write_seqcount_end(&cd.seq);
r = rate;
if (r >= 4000000) {
r /= 1000000;
r_unit = 'M';
} else if (r >= 1000) {
r /= 1000;
r_unit = 'k';
} else
r_unit = ' ';
/* calculate the ns resolution of this counter */
res = cyc_to_ns(1ULL, new_mult, new_shift);
pr_info("sched_clock: %u bits at %lu%cHz, resolution %lluns, wraps every %lluns\n",
bits, r, r_unit, res, wrap);
/* Enable IRQ time accounting if we have a fast enough sched_clock */
if (irqtime > 0 || (irqtime == -1 && rate >= 1000000))
enable_sched_clock_irqtime();
pr_debug("Registered %pF as sched_clock source\n", read);
}
int sched_clock_initialized(void)
{
return initialized;
}
void __init sched_clock_postinit(void)
{
/*
* If no sched_clock function has been provided at that point,
* make it the final one one.
*/
if (read_sched_clock == jiffy_sched_clock_read)
sched_clock_register(jiffy_sched_clock_read, BITS_PER_LONG, HZ);
update_sched_clock();
/*
* Start the timer to keep sched_clock() properly updated and
* sets the initial epoch.
*/
hrtimer_init(&sched_clock_timer, CLOCK_MONOTONIC, HRTIMER_MODE_REL);
sched_clock_timer.function = sched_clock_poll;
hrtimer_start(&sched_clock_timer, cd.wrap_kt, HRTIMER_MODE_REL);
initialized = 1;
}
static int sched_clock_suspend(void)
{
update_sched_clock();
hrtimer_cancel(&sched_clock_timer);
cd.suspended = true;
return 0;
}
static void sched_clock_resume(void)
{
cd.epoch_cyc = read_sched_clock();
hrtimer_start(&sched_clock_timer, cd.wrap_kt, HRTIMER_MODE_REL);
cd.suspended = false;
}
static struct syscore_ops sched_clock_ops = {
.suspend = sched_clock_suspend,
.resume = sched_clock_resume,
};
static int __init sched_clock_syscore_init(void)
{
register_syscore_ops(&sched_clock_ops);
return 0;
}
device_initcall(sched_clock_syscore_init);