Piteball
/
android_kernel_samsung_msm8976
mirror of https://github.com/team-infusion-developers/android_kernel_samsung_msm8976.git
1
1
Fork 0
Code Issues Projects Releases Wiki Activity

Merge branch 'perf-uprobes-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip 

Pull user-space probe instrumentation from Ingo Molnar:
 "The uprobes code originates from SystemTap and has been used for years
  in Fedora and RHEL kernels.  This version is much rewritten, reviews
  from PeterZ, Oleg and myself shaped the end result.

  This tree includes uprobes support in 'perf probe' - but SystemTap
  (and other tools) can take advantage of user probe points as well.

  Sample usage of uprobes via perf, for example to profile malloc()
  calls without modifying user-space binaries.

  First boot a new kernel with CONFIG_UPROBE_EVENT=y enabled.

  If you don't know which function you want to probe you can pick one
  from 'perf top' or can get a list all functions that can be probed
  within libc (binaries can be specified as well):

	$ perf probe -F -x /lib/libc.so.6

  To probe libc's malloc():

	$ perf probe -x /lib64/libc.so.6 malloc
	Added new event:
	probe_libc:malloc    (on 0x7eac0)

  You can now use it in all perf tools, such as:

	perf record -e probe_libc:malloc -aR sleep 1

  Make use of it to create a call graph (as the flat profile is going to
  look very boring):

	$ perf record -e probe_libc:malloc -gR make
	[ perf record: Woken up 173 times to write data ]
	[ perf record: Captured and wrote 44.190 MB perf.data (~1930712

	$ perf report | less

	  32.03%            git  libc-2.15.so   [.] malloc
	                    |
	                    --- malloc

	  29.49%            cc1  libc-2.15.so   [.] malloc
	                    |
	                    --- malloc
	                       |
	                       |--0.95%-- 0x208eb1000000000
	                       |
	                       |--0.63%-- htab_traverse_noresize

	  11.04%             as  libc-2.15.so   [.] malloc
	                     |
	                     --- malloc
	                        |

	   7.15%             ld  libc-2.15.so   [.] malloc
	                     |
	                     --- malloc
	                        |

	   5.07%             sh  libc-2.15.so   [.] malloc
	                     |
	                     --- malloc
	                        |
	   4.99%  python-config  libc-2.15.so   [.] malloc
	          |
	          --- malloc
	             |
	   4.54%           make  libc-2.15.so   [.] malloc
	                   |
	                   --- malloc
	                      |
	                      |--7.34%-- glob
	                      |          |
	                      |          |--93.18%-- 0x41588f
	                      |          |
	                      |           --6.82%-- glob
	                      |                     0x41588f

	   ...

  Or:

	$ perf report -g flat | less

	# Overhead        Command  Shared Object      Symbol
	# ........  .............  .............  ..........
	#
	  32.03%            git  libc-2.15.so   [.] malloc
	          27.19%
	              malloc

	  29.49%            cc1  libc-2.15.so   [.] malloc
	          24.77%
	              malloc

	  11.04%             as  libc-2.15.so   [.] malloc
	          11.02%
	              malloc

	   7.15%             ld  libc-2.15.so   [.] malloc
	           6.57%
	              malloc

	 ...

  The core uprobes design is fairly straightforward: uprobes probe
  points register themselves at (inode:offset) addresses of
  libraries/binaries, after which all existing (or new) vmas that map
  that address will have a software breakpoint injected at that address.
  vmas are COW-ed to preserve original content.  The probe points are
  kept in an rbtree.

  If user-space executes the probed inode:offset instruction address
  then an event is generated which can be recovered from the regular
  perf event channels and mmap-ed ring-buffer.

  Multiple probes at the same address are supported, they create a
  dynamic callback list of event consumers.

  The basic model is further complicated by the XOL speedup: the
  original instruction that is probed is copied (in an architecture
  specific fashion) and executed out of line when the probe triggers.
  The XOL area is a single vma per process, with a fixed number of
  entries (which limits probe execution parallelism).

  The API: uprobes are installed/removed via
  /sys/kernel/debug/tracing/uprobe_events, the API is integrated to
  align with the kprobes interface as much as possible, but is separate
  to it.

  Injecting a probe point is privileged operation, which can be relaxed
  by setting perf_paranoid to -1.

  You can use multiple probes as well and mix them with kprobes and
  regular PMU events or tracepoints, when instrumenting a task."

Fix up trivial conflicts in mm/memory.c due to previous cleanup of
unmap_single_vma().

* 'perf-uprobes-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip: (21 commits)
  perf probe: Detect probe target when m/x options are absent
  perf probe: Provide perf interface for uprobes
  tracing: Fix kconfig warning due to a typo
  tracing: Provide trace events interface for uprobes
  tracing: Extract out common code for kprobes/uprobes trace events
  tracing: Modify is_delete, is_return from int to bool
  uprobes/core: Decrement uprobe count before the pages are unmapped
  uprobes/core: Make background page replacement logic account for rss_stat counters
  uprobes/core: Optimize probe hits with the help of a counter
  uprobes/core: Allocate XOL slots for uprobes use
  uprobes/core: Handle breakpoint and singlestep exceptions
  uprobes/core: Rename bkpt to swbp
  uprobes/core: Make order of function parameters consistent across functions
  uprobes/core: Make macro names consistent
  uprobes: Update copyright notices
  uprobes/core: Move insn to arch specific structure
  uprobes/core: Remove uprobe_opcode_sz
  uprobes/core: Make instruction tables volatile
  uprobes: Move to kernel/events/
  uprobes/core: Clean up, refactor and improve the code
  ...
This commit is contained in:
Linus Torvalds 2012-05-24 11:39:34 -07:00
parent 2c01e7bc46 9cba26e66d
commit 654443e20d
30 changed files with 5053 additions and 984 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

113
Documentation/trace/uprobetracer.txt Normal file
View File

@ -0,0 +1,113 @@
Uprobe-tracer: Uprobe-based Event Tracing
=========================================
Documentation written by Srikar Dronamraju
Overview
--------
Uprobe based trace events are similar to kprobe based trace events.
To enable this feature, build your kernel with CONFIG_UPROBE_EVENT=y.
Similar to the kprobe-event tracer, this doesn't need to be activated via
current_tracer. Instead of that, add probe points via
/sys/kernel/debug/tracing/uprobe_events, and enable it via
/sys/kernel/debug/tracing/events/uprobes/<EVENT>/enabled.
However unlike kprobe-event tracer, the uprobe event interface expects the
user to calculate the offset of the probepoint in the object
Synopsis of uprobe_tracer
-------------------------
p[:[GRP/]EVENT] PATH:SYMBOL[+offs] [FETCHARGS] : Set a probe
GRP : Group name. If omitted, use "uprobes" for it.
EVENT : Event name. If omitted, the event name is generated
based on SYMBOL+offs.
PATH : path to an executable or a library.
SYMBOL[+offs] : Symbol+offset where the probe is inserted.
FETCHARGS : Arguments. Each probe can have up to 128 args.
%REG : Fetch register REG
Event Profiling
---------------
You can check the total number of probe hits and probe miss-hits via
/sys/kernel/debug/tracing/uprobe_profile.
The first column is event name, the second is the number of probe hits,
the third is the number of probe miss-hits.
Usage examples
--------------
To add a probe as a new event, write a new definition to uprobe_events
as below.
echo 'p: /bin/bash:0x4245c0' > /sys/kernel/debug/tracing/uprobe_events
This sets a uprobe at an offset of 0x4245c0 in the executable /bin/bash
echo > /sys/kernel/debug/tracing/uprobe_events
This clears all probe points.
The following example shows how to dump the instruction pointer and %ax
a register at the probed text address. Here we are trying to probe
function zfree in /bin/zsh
# cd /sys/kernel/debug/tracing/
# cat /proc/`pgrep zsh`/maps | grep /bin/zsh | grep r-xp
00400000-0048a000 r-xp 00000000 08:03 130904 /bin/zsh
# objdump -T /bin/zsh | grep -w zfree
0000000000446420 g DF .text 0000000000000012 Base zfree
0x46420 is the offset of zfree in object /bin/zsh that is loaded at
0x00400000. Hence the command to probe would be :
# echo 'p /bin/zsh:0x46420 %ip %ax' > uprobe_events
Please note: User has to explicitly calculate the offset of the probepoint
in the object. We can see the events that are registered by looking at the
uprobe_events file.
# cat uprobe_events
p:uprobes/p_zsh_0x46420 /bin/zsh:0x00046420 arg1=%ip arg2=%ax
The format of events can be seen by viewing the file events/uprobes/p_zsh_0x46420/format
# cat events/uprobes/p_zsh_0x46420/format
name: p_zsh_0x46420
ID: 922
format:
field:unsigned short common_type; offset:0; size:2; signed:0;
field:unsigned char common_flags; offset:2; size:1; signed:0;
field:unsigned char common_preempt_count; offset:3; size:1; signed:0;
field:int common_pid; offset:4; size:4; signed:1;
field:int common_padding; offset:8; size:4; signed:1;
field:unsigned long __probe_ip; offset:12; size:4; signed:0;
field:u32 arg1; offset:16; size:4; signed:0;
field:u32 arg2; offset:20; size:4; signed:0;
print fmt: "(%lx) arg1=%lx arg2=%lx", REC->__probe_ip, REC->arg1, REC->arg2
Right after definition, each event is disabled by default. For tracing these
events, you need to enable it by:
# echo 1 > events/uprobes/enable
Lets disable the event after sleeping for some time.
# sleep 20
# echo 0 > events/uprobes/enable
And you can see the traced information via /sys/kernel/debug/tracing/trace.
# cat trace
# tracer: nop
#
# TASK-PID CPU# TIMESTAMP FUNCTION
# | | | | |
zsh-24842 [006] 258544.995456: p_zsh_0x46420: (0x446420) arg1=446421 arg2=79
zsh-24842 [007] 258545.000270: p_zsh_0x46420: (0x446420) arg1=446421 arg2=79
zsh-24842 [002] 258545.043929: p_zsh_0x46420: (0x446420) arg1=446421 arg2=79
zsh-24842 [004] 258547.046129: p_zsh_0x46420: (0x446420) arg1=446421 arg2=79
Each line shows us probes were triggered for a pid 24842 with ip being
0x446421 and contents of ax register being 79.

17
arch/Kconfig
View File

@ -76,6 +76,23 @@ config OPTPROBES
depends on KPROBES && HAVE_OPTPROBES
depends on !PREEMPT
config UPROBES
bool "Transparent user-space probes (EXPERIMENTAL)"
depends on UPROBE_EVENT && PERF_EVENTS
default n
help
Uprobes is the user-space counterpart to kprobes: they
enable instrumentation applications (such as 'perf probe')
to establish unintrusive probes in user-space binaries and
libraries, by executing handler functions when the probes
are hit by user-space applications.
( These probes come in the form of single-byte breakpoints,
managed by the kernel and kept transparent to the probed
application. )
If in doubt, say "N".
config HAVE_EFFICIENT_UNALIGNED_ACCESS
bool
help

5
arch/x86/Kconfig
View File

@ -87,7 +87,7 @@ config X86
select BUILDTIME_EXTABLE_SORT
config INSTRUCTION_DECODER
def_bool (KPROBES || PERF_EVENTS)
def_bool (KPROBES || PERF_EVENTS || UPROBES)
config OUTPUT_FORMAT
string
@ -243,6 +243,9 @@ config ARCH_CPU_PROBE_RELEASE
def_bool y
depends on HOTPLUG_CPU
config ARCH_SUPPORTS_UPROBES
def_bool y
source "init/Kconfig"
source "kernel/Kconfig.freezer"

2
arch/x86/include/asm/thread_info.h
View File

@ -85,6 +85,7 @@ struct thread_info {
#define TIF_SECCOMP 8 /* secure computing */
#define TIF_MCE_NOTIFY 10 /* notify userspace of an MCE */
#define TIF_USER_RETURN_NOTIFY 11 /* notify kernel of userspace return */
#define TIF_UPROBE 12 /* breakpointed or singlestepping */
#define TIF_NOTSC 16 /* TSC is not accessible in userland */
#define TIF_IA32 17 /* IA32 compatibility process */
#define TIF_FORK 18 /* ret_from_fork */
@ -109,6 +110,7 @@ struct thread_info {
#define _TIF_SECCOMP (1 << TIF_SECCOMP)
#define _TIF_MCE_NOTIFY (1 << TIF_MCE_NOTIFY)
#define _TIF_USER_RETURN_NOTIFY (1 << TIF_USER_RETURN_NOTIFY)
#define _TIF_UPROBE (1 << TIF_UPROBE)
#define _TIF_NOTSC (1 << TIF_NOTSC)
#define _TIF_IA32 (1 << TIF_IA32)
#define _TIF_FORK (1 << TIF_FORK)

57
arch/x86/include/asm/uprobes.h Normal file
View File

@ -0,0 +1,57 @@
#ifndef _ASM_UPROBES_H
#define _ASM_UPROBES_H
/*
* User-space Probes (UProbes) for x86
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 2 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
*
* Copyright (C) IBM Corporation, 2008-2011
* Authors:
* Srikar Dronamraju
* Jim Keniston
*/
#include <linux/notifier.h>
typedef u8 uprobe_opcode_t;
#define MAX_UINSN_BYTES 16
#define UPROBE_XOL_SLOT_BYTES 128 /* to keep it cache aligned */
#define UPROBE_SWBP_INSN 0xcc
#define UPROBE_SWBP_INSN_SIZE 1
struct arch_uprobe {
u16 fixups;
u8 insn[MAX_UINSN_BYTES];
#ifdef CONFIG_X86_64
unsigned long rip_rela_target_address;
#endif
};
struct arch_uprobe_task {
unsigned long saved_trap_nr;
#ifdef CONFIG_X86_64
unsigned long saved_scratch_register;
#endif
};
extern int arch_uprobe_analyze_insn(struct arch_uprobe *aup, struct mm_struct *mm);
extern int arch_uprobe_pre_xol(struct arch_uprobe *aup, struct pt_regs *regs);
extern int arch_uprobe_post_xol(struct arch_uprobe *aup, struct pt_regs *regs);
extern bool arch_uprobe_xol_was_trapped(struct task_struct *tsk);
extern int arch_uprobe_exception_notify(struct notifier_block *self, unsigned long val, void *data);
extern void arch_uprobe_abort_xol(struct arch_uprobe *aup, struct pt_regs *regs);
#endif /* _ASM_UPROBES_H */

1
arch/x86/kernel/Makefile
View File

@ -100,6 +100,7 @@ obj-$(CONFIG_X86_CHECK_BIOS_CORRUPTION) += check.o
obj-$(CONFIG_SWIOTLB) += pci-swiotlb.o
obj-$(CONFIG_OF) += devicetree.o
obj-$(CONFIG_UPROBES) += uprobes.o
###
# 64 bit specific files

6
arch/x86/kernel/signal.c
View File

@ -18,6 +18,7 @@
#include <linux/personality.h>
#include <linux/uaccess.h>
#include <linux/user-return-notifier.h>
#include <linux/uprobes.h>
#include <asm/processor.h>
#include <asm/ucontext.h>
@ -814,6 +815,11 @@ do_notify_resume(struct pt_regs *regs, void *unused, __u32 thread_info_flags)
mce_notify_process();
#endif /* CONFIG_X86_64 && CONFIG_X86_MCE */
if (thread_info_flags & _TIF_UPROBE) {
clear_thread_flag(TIF_UPROBE);
uprobe_notify_resume(regs);
}
/* deal with pending signal delivery */
if (thread_info_flags & _TIF_SIGPENDING)
do_signal(regs);

674
arch/x86/kernel/uprobes.c Normal file
View File

@ -0,0 +1,674 @@
/*
* User-space Probes (UProbes) for x86
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 2 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
*
* Copyright (C) IBM Corporation, 2008-2011
* Authors:
* Srikar Dronamraju
* Jim Keniston
*/
#include <linux/kernel.h>
#include <linux/sched.h>
#include <linux/ptrace.h>
#include <linux/uprobes.h>
#include <linux/uaccess.h>
#include <linux/kdebug.h>
#include <asm/processor.h>
#include <asm/insn.h>
/* Post-execution fixups. */
/* No fixup needed */
#define UPROBE_FIX_NONE 0x0
/* Adjust IP back to vicinity of actual insn */
#define UPROBE_FIX_IP 0x1
/* Adjust the return address of a call insn */
#define UPROBE_FIX_CALL 0x2
#define UPROBE_FIX_RIP_AX 0x8000
#define UPROBE_FIX_RIP_CX 0x4000
#define UPROBE_TRAP_NR UINT_MAX
/* Adaptations for mhiramat x86 decoder v14. */
#define OPCODE1(insn) ((insn)->opcode.bytes[0])
#define OPCODE2(insn) ((insn)->opcode.bytes[1])
#define OPCODE3(insn) ((insn)->opcode.bytes[2])
#define MODRM_REG(insn) X86_MODRM_REG(insn->modrm.value)
#define W(row, b0, b1, b2, b3, b4, b5, b6, b7, b8, b9, ba, bb, bc, bd, be, bf)\
(((b0##UL << 0x0)|(b1##UL << 0x1)|(b2##UL << 0x2)|(b3##UL << 0x3) | \
(b4##UL << 0x4)|(b5##UL << 0x5)|(b6##UL << 0x6)|(b7##UL << 0x7) | \
(b8##UL << 0x8)|(b9##UL << 0x9)|(ba##UL << 0xa)|(bb##UL << 0xb) | \
(bc##UL << 0xc)|(bd##UL << 0xd)|(be##UL << 0xe)|(bf##UL << 0xf)) \
<< (row % 32))
/*
* Good-instruction tables for 32-bit apps. This is non-const and volatile
* to keep gcc from statically optimizing it out, as variable_test_bit makes
* some versions of gcc to think only *(unsigned long*) is used.
*/
static volatile u32 good_insns_32[256 / 32] = {
/* 0 1 2 3 4 5 6 7 8 9 a b c d e f */
/* ---------------------------------------------- */
W(0x00, 1, 1, 1, 1, 1, 1, 1, 0, 1, 1, 1, 1, 1, 1, 1, 0) | /* 00 */
W(0x10, 1, 1, 1, 1, 1, 1, 1, 0, 1, 1, 1, 1, 1, 1, 1, 0) , /* 10 */
W(0x20, 1, 1, 1, 1, 1, 1, 0, 1, 1, 1, 1, 1, 1, 1, 0, 1) | /* 20 */
W(0x30, 1, 1, 1, 1, 1, 1, 0, 1, 1, 1, 1, 1, 1, 1, 0, 1) , /* 30 */
W(0x40, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1) | /* 40 */
W(0x50, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1) , /* 50 */
W(0x60, 1, 1, 1, 0, 1, 1, 0, 0, 1, 1, 1, 1, 0, 0, 0, 0) | /* 60 */
W(0x70, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1) , /* 70 */
W(0x80, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1) | /* 80 */
W(0x90, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1) , /* 90 */
W(0xa0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1) | /* a0 */
W(0xb0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1) , /* b0 */
W(0xc0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0) | /* c0 */
W(0xd0, 1, 1, 1, 1, 1, 1, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1) , /* d0 */
W(0xe0, 1, 1, 1, 1, 0, 0, 0, 0, 1, 1, 1, 1, 0, 0, 0, 0) | /* e0 */
W(0xf0, 0, 0, 1, 1, 0, 1, 1, 1, 1, 1, 0, 0, 1, 1, 1, 1) /* f0 */
/* ---------------------------------------------- */
/* 0 1 2 3 4 5 6 7 8 9 a b c d e f */
};
/* Using this for both 64-bit and 32-bit apps */
static volatile u32 good_2byte_insns[256 / 32] = {
/* 0 1 2 3 4 5 6 7 8 9 a b c d e f */
/* ---------------------------------------------- */
W(0x00, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1) | /* 00 */
W(0x10, 1, 1, 1, 1, 1, 1, 1, 1, 0, 1, 1, 1, 1, 1, 1, 1) , /* 10 */
W(0x20, 1, 1, 1, 1, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1) | /* 20 */
W(0x30, 0, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0) , /* 30 */
W(0x40, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1) | /* 40 */
W(0x50, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1) , /* 50 */
W(0x60, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1) | /* 60 */
W(0x70, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 1, 1) , /* 70 */
W(0x80, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1) | /* 80 */
W(0x90, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1) , /* 90 */
W(0xa0, 1, 1, 1, 1, 1, 1, 0, 0, 1, 1, 1, 1, 1, 1, 0, 1) | /* a0 */
W(0xb0, 1, 1, 1, 1, 1, 1, 1, 1, 0, 1, 1, 1, 1, 1, 1, 1) , /* b0 */
W(0xc0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1) | /* c0 */
W(0xd0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1) , /* d0 */
W(0xe0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1) | /* e0 */
W(0xf0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0) /* f0 */
/* ---------------------------------------------- */
/* 0 1 2 3 4 5 6 7 8 9 a b c d e f */
};
#ifdef CONFIG_X86_64
/* Good-instruction tables for 64-bit apps */
static volatile u32 good_insns_64[256 / 32] = {
/* 0 1 2 3 4 5 6 7 8 9 a b c d e f */
/* ---------------------------------------------- */
W(0x00, 1, 1, 1, 1, 1, 1, 0, 0, 1, 1, 1, 1, 1, 1, 0, 0) | /* 00 */
W(0x10, 1, 1, 1, 1, 1, 1, 0, 0, 1, 1, 1, 1, 1, 1, 0, 0) , /* 10 */
W(0x20, 1, 1, 1, 1, 1, 1, 0, 0, 1, 1, 1, 1, 1, 1, 0, 0) | /* 20 */
W(0x30, 1, 1, 1, 1, 1, 1, 0, 0, 1, 1, 1, 1, 1, 1, 0, 0) , /* 30 */
W(0x40, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0) | /* 40 */
W(0x50, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1) , /* 50 */
W(0x60, 0, 0, 0, 1, 1, 1, 0, 0, 1, 1, 1, 1, 0, 0, 0, 0) | /* 60 */
W(0x70, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1) , /* 70 */
W(0x80, 1, 1, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1) | /* 80 */
W(0x90, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1) , /* 90 */
W(0xa0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1) | /* a0 */
W(0xb0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1) , /* b0 */
W(0xc0, 1, 1, 1, 1, 0, 0, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0) | /* c0 */
W(0xd0, 1, 1, 1, 1, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1) , /* d0 */
W(0xe0, 1, 1, 1, 1, 0, 0, 0, 0, 1, 1, 1, 1, 0, 0, 0, 0) | /* e0 */
W(0xf0, 0, 0, 1, 1, 0, 1, 1, 1, 1, 1, 0, 0, 1, 1, 1, 1) /* f0 */
/* ---------------------------------------------- */
/* 0 1 2 3 4 5 6 7 8 9 a b c d e f */
};
#endif
#undef W
/*
* opcodes we'll probably never support:
*
* 6c-6d, e4-e5, ec-ed - in
* 6e-6f, e6-e7, ee-ef - out
* cc, cd - int3, int
* cf - iret
* d6 - illegal instruction
* f1 - int1/icebp
* f4 - hlt
* fa, fb - cli, sti
* 0f - lar, lsl, syscall, clts, sysret, sysenter, sysexit, invd, wbinvd, ud2
*
* invalid opcodes in 64-bit mode:
*
* 06, 0e, 16, 1e, 27, 2f, 37, 3f, 60-62, 82, c4-c5, d4-d5
* 63 - we support this opcode in x86_64 but not in i386.
*
* opcodes we may need to refine support for:
*
* 0f - 2-byte instructions: For many of these instructions, the validity
* depends on the prefix and/or the reg field. On such instructions, we
* just consider the opcode combination valid if it corresponds to any
* valid instruction.
*
* 8f - Group 1 - only reg = 0 is OK
* c6-c7 - Group 11 - only reg = 0 is OK
* d9-df - fpu insns with some illegal encodings
* f2, f3 - repnz, repz prefixes. These are also the first byte for
* certain floating-point instructions, such as addsd.
*
* fe - Group 4 - only reg = 0 or 1 is OK
* ff - Group 5 - only reg = 0-6 is OK
*
* others -- Do we need to support these?
*
* 0f - (floating-point?) prefetch instructions
* 07, 17, 1f - pop es, pop ss, pop ds
* 26, 2e, 36, 3e - es:, cs:, ss:, ds: segment prefixes --
* but 64 and 65 (fs: and gs:) seem to be used, so we support them
* 67 - addr16 prefix
* ce - into
* f0 - lock prefix
*/
/*
* TODO:
* - Where necessary, examine the modrm byte and allow only valid instructions
* in the different Groups and fpu instructions.
*/
static bool is_prefix_bad(struct insn *insn)
{
int i;
for (i = 0; i < insn->prefixes.nbytes; i++) {
switch (insn->prefixes.bytes[i]) {
case 0x26: /* INAT_PFX_ES */
case 0x2E: /* INAT_PFX_CS */
case 0x36: /* INAT_PFX_DS */
case 0x3E: /* INAT_PFX_SS */
case 0xF0: /* INAT_PFX_LOCK */
return true;
}
}
return false;
}
static int validate_insn_32bits(struct arch_uprobe *auprobe, struct insn *insn)
{
insn_init(insn, auprobe->insn, false);
/* Skip good instruction prefixes; reject "bad" ones. */
insn_get_opcode(insn);
if (is_prefix_bad(insn))
return -ENOTSUPP;
if (test_bit(OPCODE1(insn), (unsigned long *)good_insns_32))
return 0;
if (insn->opcode.nbytes == 2) {
if (test_bit(OPCODE2(insn), (unsigned long *)good_2byte_insns))
return 0;
}
return -ENOTSUPP;
}
/*
* Figure out which fixups arch_uprobe_post_xol() will need to perform, and
* annotate arch_uprobe->fixups accordingly. To start with,
* arch_uprobe->fixups is either zero or it reflects rip-related fixups.
*/
static void prepare_fixups(struct arch_uprobe *auprobe, struct insn *insn)
{
bool fix_ip = true, fix_call = false; /* defaults */
int reg;
insn_get_opcode(insn); /* should be a nop */
switch (OPCODE1(insn)) {
case 0xc3: /* ret/lret */
case 0xcb:
case 0xc2:
case 0xca:
/* ip is correct */
fix_ip = false;
break;
case 0xe8: /* call relative - Fix return addr */
fix_call = true;
break;
case 0x9a: /* call absolute - Fix return addr, not ip */
fix_call = true;
fix_ip = false;
break;
case 0xff:
insn_get_modrm(insn);
reg = MODRM_REG(insn);
if (reg == 2 || reg == 3) {
/* call or lcall, indirect */
/* Fix return addr; ip is correct. */
fix_call = true;
fix_ip = false;
} else if (reg == 4 || reg == 5) {
/* jmp or ljmp, indirect */
/* ip is correct. */
fix_ip = false;
}
break;
case 0xea: /* jmp absolute -- ip is correct */
fix_ip = false;
break;
default:
break;
}
if (fix_ip)
auprobe->fixups |= UPROBE_FIX_IP;
if (fix_call)
auprobe->fixups |= UPROBE_FIX_CALL;
}
#ifdef CONFIG_X86_64
/*
* If arch_uprobe->insn doesn't use rip-relative addressing, return
* immediately. Otherwise, rewrite the instruction so that it accesses
* its memory operand indirectly through a scratch register. Set
* arch_uprobe->fixups and arch_uprobe->rip_rela_target_address
* accordingly. (The contents of the scratch register will be saved
* before we single-step the modified instruction, and restored
* afterward.)
*
* We do this because a rip-relative instruction can access only a
* relatively small area (+/- 2 GB from the instruction), and the XOL
* area typically lies beyond that area. At least for instructions
* that store to memory, we can't execute the original instruction
* and "fix things up" later, because the misdirected store could be
* disastrous.
*
* Some useful facts about rip-relative instructions:
*
* - There's always a modrm byte.
* - There's never a SIB byte.
* - The displacement is always 4 bytes.
*/
static void
handle_riprel_insn(struct arch_uprobe *auprobe, struct mm_struct *mm, struct insn *insn)
{
u8 *cursor;
u8 reg;
if (mm->context.ia32_compat)
return;
auprobe->rip_rela_target_address = 0x0;
if (!insn_rip_relative(insn))
return;
/*
* insn_rip_relative() would have decoded rex_prefix, modrm.
* Clear REX.b bit (extension of MODRM.rm field):
* we want to encode rax/rcx, not r8/r9.
*/
if (insn->rex_prefix.nbytes) {
cursor = auprobe->insn + insn_offset_rex_prefix(insn);
*cursor &= 0xfe; /* Clearing REX.B bit */
}
/*
* Point cursor at the modrm byte. The next 4 bytes are the
* displacement. Beyond the displacement, for some instructions,
* is the immediate operand.
*/
cursor = auprobe->insn + insn_offset_modrm(insn);
insn_get_length(insn);
/*
* Convert from rip-relative addressing to indirect addressing
* via a scratch register. Change the r/m field from 0x5 (%rip)
* to 0x0 (%rax) or 0x1 (%rcx), and squeeze out the offset field.
*/
reg = MODRM_REG(insn);
if (reg == 0) {
/*
* The register operand (if any) is either the A register
* (%rax, %eax, etc.) or (if the 0x4 bit is set in the
* REX prefix) %r8. In any case, we know the C register
* is NOT the register operand, so we use %rcx (register
* #1) for the scratch register.
*/
auprobe->fixups = UPROBE_FIX_RIP_CX;
/* Change modrm from 00 000 101 to 00 000 001. */
*cursor = 0x1;
} else {
/* Use %rax (register #0) for the scratch register. */
auprobe->fixups = UPROBE_FIX_RIP_AX;
/* Change modrm from 00 xxx 101 to 00 xxx 000 */
*cursor = (reg << 3);
}
/* Target address = address of next instruction + (signed) offset */
auprobe->rip_rela_target_address = (long)insn->length + insn->displacement.value;
/* Displacement field is gone; slide immediate field (if any) over. */
if (insn->immediate.nbytes) {
cursor++;
memmove(cursor, cursor + insn->displacement.nbytes, insn->immediate.nbytes);
}
return;
}
static int validate_insn_64bits(struct arch_uprobe *auprobe, struct insn *insn)
{
insn_init(insn, auprobe->insn, true);
/* Skip good instruction prefixes; reject "bad" ones. */
insn_get_opcode(insn);
if (is_prefix_bad(insn))
return -ENOTSUPP;
if (test_bit(OPCODE1(insn), (unsigned long *)good_insns_64))
return 0;
if (insn->opcode.nbytes == 2) {
if (test_bit(OPCODE2(insn), (unsigned long *)good_2byte_insns))
return 0;
}
return -ENOTSUPP;
}
static int validate_insn_bits(struct arch_uprobe *auprobe, struct mm_struct *mm, struct insn *insn)
{
if (mm->context.ia32_compat)
return validate_insn_32bits(auprobe, insn);
return validate_insn_64bits(auprobe, insn);
}
#else /* 32-bit: */
static void handle_riprel_insn(struct arch_uprobe *auprobe, struct mm_struct *mm, struct insn *insn)
{
/* No RIP-relative addressing on 32-bit */
}
static int validate_insn_bits(struct arch_uprobe *auprobe, struct mm_struct *mm, struct insn *insn)
{
return validate_insn_32bits(auprobe, insn);
}
#endif /* CONFIG_X86_64 */
/**
* arch_uprobe_analyze_insn - instruction analysis including validity and fixups.
* @mm: the probed address space.
* @arch_uprobe: the probepoint information.
* Return 0 on success or a -ve number on error.
*/
int arch_uprobe_analyze_insn(struct arch_uprobe *auprobe, struct mm_struct *mm)
{
int ret;
struct insn insn;
auprobe->fixups = 0;
ret = validate_insn_bits(auprobe, mm, &insn);
if (ret != 0)
return ret;
handle_riprel_insn(auprobe, mm, &insn);
prepare_fixups(auprobe, &insn);
return 0;
}
#ifdef CONFIG_X86_64
/*
* If we're emulating a rip-relative instruction, save the contents
* of the scratch register and store the target address in that register.
*/
static void
pre_xol_rip_insn(struct arch_uprobe *auprobe, struct pt_regs *regs,
struct arch_uprobe_task *autask)
{
if (auprobe->fixups & UPROBE_FIX_RIP_AX) {
autask->saved_scratch_register = regs->ax;
regs->ax = current->utask->vaddr;
regs->ax += auprobe->rip_rela_target_address;
} else if (auprobe->fixups & UPROBE_FIX_RIP_CX) {
autask->saved_scratch_register = regs->cx;
regs->cx = current->utask->vaddr;
regs->cx += auprobe->rip_rela_target_address;
}
}
#else
static void
pre_xol_rip_insn(struct arch_uprobe *auprobe, struct pt_regs *regs,
struct arch_uprobe_task *autask)
{
/* No RIP-relative addressing on 32-bit */
}
#endif
/*
* arch_uprobe_pre_xol - prepare to execute out of line.
* @auprobe: the probepoint information.
* @regs: reflects the saved user state of current task.
*/
int arch_uprobe_pre_xol(struct arch_uprobe *auprobe, struct pt_regs *regs)
{
struct arch_uprobe_task *autask;
autask = &current->utask->autask;
autask->saved_trap_nr = current->thread.trap_nr;
current->thread.trap_nr = UPROBE_TRAP_NR;
regs->ip = current->utask->xol_vaddr;
pre_xol_rip_insn(auprobe, regs, autask);
return 0;
}
/*
* This function is called by arch_uprobe_post_xol() to adjust the return
* address pushed by a call instruction executed out of line.
*/
static int adjust_ret_addr(unsigned long sp, long correction)
{
int rasize, ncopied;
long ra = 0;
if (is_ia32_task())
rasize = 4;
else
rasize = 8;
ncopied = copy_from_user(&ra, (void __user *)sp, rasize);
if (unlikely(ncopied))
return -EFAULT;
ra += correction;
ncopied = copy_to_user((void __user *)sp, &ra, rasize);
if (unlikely(ncopied))
return -EFAULT;
return 0;
}
#ifdef CONFIG_X86_64
static bool is_riprel_insn(struct arch_uprobe *auprobe)
{
return ((auprobe->fixups & (UPROBE_FIX_RIP_AX | UPROBE_FIX_RIP_CX)) != 0);
}
static void
handle_riprel_post_xol(struct arch_uprobe *auprobe, struct pt_regs *regs, long *correction)
{
if (is_riprel_insn(auprobe)) {
struct arch_uprobe_task *autask;
autask = &current->utask->autask;
if (auprobe->fixups & UPROBE_FIX_RIP_AX)
regs->ax = autask->saved_scratch_register;
else
regs->cx = autask->saved_scratch_register;
/*
* The original instruction includes a displacement, and so
* is 4 bytes longer than what we've just single-stepped.
* Fall through to handle stuff like "jmpq *...(%rip)" and
* "callq *...(%rip)".
*/
if (correction)
*correction += 4;
}
}
#else
static void
handle_riprel_post_xol(struct arch_uprobe *auprobe, struct pt_regs *regs, long *correction)
{
/* No RIP-relative addressing on 32-bit */
}
#endif
/*
* If xol insn itself traps and generates a signal(Say,
* SIGILL/SIGSEGV/etc), then detect the case where a singlestepped
* instruction jumps back to its own address. It is assumed that anything
* like do_page_fault/do_trap/etc sets thread.trap_nr != -1.
*
* arch_uprobe_pre_xol/arch_uprobe_post_xol save/restore thread.trap_nr,
* arch_uprobe_xol_was_trapped() simply checks that ->trap_nr is not equal to
* UPROBE_TRAP_NR == -1 set by arch_uprobe_pre_xol().
*/
bool arch_uprobe_xol_was_trapped(struct task_struct *t)
{
if (t->thread.trap_nr != UPROBE_TRAP_NR)
return true;
return false;
}
/*
* Called after single-stepping. To avoid the SMP problems that can
* occur when we temporarily put back the original opcode to
* single-step, we single-stepped a copy of the instruction.
*
* This function prepares to resume execution after the single-step.
* We have to fix things up as follows:
*
* Typically, the new ip is relative to the copied instruction. We need
* to make it relative to the original instruction (FIX_IP). Exceptions
* are return instructions and absolute or indirect jump or call instructions.
*
* If the single-stepped instruction was a call, the return address that
* is atop the stack is the address following the copied instruction. We
* need to make it the address following the original instruction (FIX_CALL).
*
* If the original instruction was a rip-relative instruction such as
* "movl %edx,0xnnnn(%rip)", we have instead executed an equivalent
* instruction using a scratch register -- e.g., "movl %edx,(%rax)".
* We need to restore the contents of the scratch register and adjust
* the ip, keeping in mind that the instruction we executed is 4 bytes
* shorter than the original instruction (since we squeezed out the offset
* field). (FIX_RIP_AX or FIX_RIP_CX)
*/
int arch_uprobe_post_xol(struct arch_uprobe *auprobe, struct pt_regs *regs)
{
struct uprobe_task *utask;
long correction;
int result = 0;
WARN_ON_ONCE(current->thread.trap_nr != UPROBE_TRAP_NR);
utask = current->utask;
current->thread.trap_nr = utask->autask.saved_trap_nr;
correction = (long)(utask->vaddr - utask->xol_vaddr);
handle_riprel_post_xol(auprobe, regs, &correction);
if (auprobe->fixups & UPROBE_FIX_IP)
regs->ip += correction;
if (auprobe->fixups & UPROBE_FIX_CALL)
result = adjust_ret_addr(regs->sp, correction);
return result;
}
/* callback routine for handling exceptions. */
int arch_uprobe_exception_notify(struct notifier_block *self, unsigned long val, void *data)
{
struct die_args *args = data;
struct pt_regs *regs = args->regs;
int ret = NOTIFY_DONE;
/* We are only interested in userspace traps */
if (regs && !user_mode_vm(regs))
return NOTIFY_DONE;
switch (val) {
case DIE_INT3:
if (uprobe_pre_sstep_notifier(regs))
ret = NOTIFY_STOP;
break;
case DIE_DEBUG:
if (uprobe_post_sstep_notifier(regs))
ret = NOTIFY_STOP;
default:
break;
}
return ret;
}
/*
* This function gets called when XOL instruction either gets trapped or
* the thread has a fatal signal, so reset the instruction pointer to its
* probed address.
*/
void arch_uprobe_abort_xol(struct arch_uprobe *auprobe, struct pt_regs *regs)
{
struct uprobe_task *utask = current->utask;
current->thread.trap_nr = utask->autask.saved_trap_nr;
handle_riprel_post_xol(auprobe, regs, NULL);
instruction_pointer_set(regs, utask->vaddr);
}
/*
* Skip these instructions as per the currently known x86 ISA.
* 0x66* { 0x90 | 0x0f 0x1f | 0x0f 0x19 | 0x87 0xc0 }
*/
bool arch_uprobe_skip_sstep(struct arch_uprobe *auprobe, struct pt_regs *regs)
{
int i;
for (i = 0; i < MAX_UINSN_BYTES; i++) {
if ((auprobe->insn[i] == 0x66))
continue;
if (auprobe->insn[i] == 0x90)
return true;
if (i == (MAX_UINSN_BYTES - 1))
break;
if ((auprobe->insn[i] == 0x0f) && (auprobe->insn[i+1] == 0x1f))
return true;
if ((auprobe->insn[i] == 0x0f) && (auprobe->insn[i+1] == 0x19))
return true;
if ((auprobe->insn[i] == 0x87) && (auprobe->insn[i+1] == 0xc0))
return true;
break;
}
return false;
}

2
include/linux/mm_types.h
View File

@ -12,6 +12,7 @@
#include <linux/completion.h>
#include <linux/cpumask.h>
#include <linux/page-debug-flags.h>
#include <linux/uprobes.h>
#include <asm/page.h>
#include <asm/mmu.h>
@ -388,6 +389,7 @@ struct mm_struct {
#ifdef CONFIG_CPUMASK_OFFSTACK
struct cpumask cpumask_allocation;
#endif
struct uprobes_state uprobes_state;
};
static inline void mm_init_cpumask(struct mm_struct *mm)

4
include/linux/sched.h
View File

@ -1572,6 +1572,10 @@ struct task_struct {
#ifdef CONFIG_HAVE_HW_BREAKPOINT
atomic_t ptrace_bp_refcnt;
#endif
#ifdef CONFIG_UPROBES
struct uprobe_task *utask;
int uprobe_srcu_id;
#endif
};
/* Future-safe accessor for struct task_struct's cpus_allowed. */

165
include/linux/uprobes.h Normal file
View File

@ -0,0 +1,165 @@
#ifndef _LINUX_UPROBES_H
#define _LINUX_UPROBES_H
/*
* User-space Probes (UProbes)
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 2 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
*
* Copyright (C) IBM Corporation, 2008-2012
* Authors:
* Srikar Dronamraju
* Jim Keniston
* Copyright (C) 2011-2012 Red Hat, Inc., Peter Zijlstra <pzijlstr@redhat.com>
*/
#include <linux/errno.h>
#include <linux/rbtree.h>
struct vm_area_struct;
struct mm_struct;
struct inode;
#ifdef CONFIG_ARCH_SUPPORTS_UPROBES
# include <asm/uprobes.h>
#endif
/* flags that denote/change uprobes behaviour */
/* Have a copy of original instruction */
#define UPROBE_COPY_INSN 0x1
/* Dont run handlers when first register/ last unregister in progress*/
#define UPROBE_RUN_HANDLER 0x2
/* Can skip singlestep */
#define UPROBE_SKIP_SSTEP 0x4
struct uprobe_consumer {
int (*handler)(struct uprobe_consumer *self, struct pt_regs *regs);
/*
* filter is optional; If a filter exists, handler is run
* if and only if filter returns true.
*/
bool (*filter)(struct uprobe_consumer *self, struct task_struct *task);
struct uprobe_consumer *next;
};
#ifdef CONFIG_UPROBES
enum uprobe_task_state {
UTASK_RUNNING,
UTASK_BP_HIT,
UTASK_SSTEP,
UTASK_SSTEP_ACK,
UTASK_SSTEP_TRAPPED,
};
/*
* uprobe_task: Metadata of a task while it singlesteps.
*/
struct uprobe_task {
enum uprobe_task_state state;
struct arch_uprobe_task autask;
struct uprobe *active_uprobe;
unsigned long xol_vaddr;
unsigned long vaddr;
};
/*
* On a breakpoint hit, thread contests for a slot. It frees the
* slot after singlestep. Currently a fixed number of slots are
* allocated.
*/
struct xol_area {
wait_queue_head_t wq; /* if all slots are busy */
atomic_t slot_count; /* number of in-use slots */
unsigned long *bitmap; /* 0 = free slot */
struct page *page;
/*
* We keep the vma's vm_start rather than a pointer to the vma
* itself. The probed process or a naughty kernel module could make
* the vma go away, and we must handle that reasonably gracefully.
*/
unsigned long vaddr; /* Page(s) of instruction slots */
};
struct uprobes_state {
struct xol_area *xol_area;
atomic_t count;
};
extern int __weak set_swbp(struct arch_uprobe *aup, struct mm_struct *mm, unsigned long vaddr);
extern int __weak set_orig_insn(struct arch_uprobe *aup, struct mm_struct *mm, unsigned long vaddr, bool verify);
extern bool __weak is_swbp_insn(uprobe_opcode_t *insn);
extern int uprobe_register(struct inode *inode, loff_t offset, struct uprobe_consumer *uc);
extern void uprobe_unregister(struct inode *inode, loff_t offset, struct uprobe_consumer *uc);
extern int uprobe_mmap(struct vm_area_struct *vma);
extern void uprobe_munmap(struct vm_area_struct *vma, unsigned long start, unsigned long end);
extern void uprobe_free_utask(struct task_struct *t);
extern void uprobe_copy_process(struct task_struct *t);
extern unsigned long __weak uprobe_get_swbp_addr(struct pt_regs *regs);
extern int uprobe_post_sstep_notifier(struct pt_regs *regs);
extern int uprobe_pre_sstep_notifier(struct pt_regs *regs);
extern void uprobe_notify_resume(struct pt_regs *regs);
extern bool uprobe_deny_signal(void);
extern bool __weak arch_uprobe_skip_sstep(struct arch_uprobe *aup, struct pt_regs *regs);
extern void uprobe_clear_state(struct mm_struct *mm);
extern void uprobe_reset_state(struct mm_struct *mm);
#else /* !CONFIG_UPROBES */
struct uprobes_state {
};
static inline int
uprobe_register(struct inode *inode, loff_t offset, struct uprobe_consumer *uc)
{
return -ENOSYS;
}
static inline void
uprobe_unregister(struct inode *inode, loff_t offset, struct uprobe_consumer *uc)
{
}
static inline int uprobe_mmap(struct vm_area_struct *vma)
{
return 0;
}
static inline void
uprobe_munmap(struct vm_area_struct *vma, unsigned long start, unsigned long end)
{
}
static inline void uprobe_notify_resume(struct pt_regs *regs)
{
}
static inline bool uprobe_deny_signal(void)
{
return false;
}
static inline unsigned long uprobe_get_swbp_addr(struct pt_regs *regs)
{
return 0;
}
static inline void uprobe_free_utask(struct task_struct *t)
{
}
static inline void uprobe_copy_process(struct task_struct *t)
{
}
static inline void uprobe_clear_state(struct mm_struct *mm)
{
}
static inline void uprobe_reset_state(struct mm_struct *mm)
{
}
#endif /* !CONFIG_UPROBES */
#endif /* _LINUX_UPROBES_H */

3
kernel/events/Makefile
View File

@ -3,4 +3,7 @@ CFLAGS_REMOVE_core.o = -pg
endif
obj-y := core.o ring_buffer.o callchain.o
obj-$(CONFIG_HAVE_HW_BREAKPOINT) += hw_breakpoint.o
obj-$(CONFIG_UPROBES) += uprobes.o

1667
kernel/events/uprobes.c Normal file
View File

File diff suppressed because it is too large Load Diff

9
kernel/fork.c
View File

@ -69,6 +69,7 @@
#include <linux/oom.h>
#include <linux/khugepaged.h>
#include <linux/signalfd.h>
#include <linux/uprobes.h>
#include <asm/pgtable.h>
#include <asm/pgalloc.h>
@ -451,6 +452,9 @@ static int dup_mmap(struct mm_struct *mm, struct mm_struct *oldmm)
if (retval)
goto out;
if (file && uprobe_mmap(tmp))
goto out;
}
/* a new mm has just been created */
arch_dup_mmap(oldmm, mm);
@ -599,6 +603,7 @@ void mmput(struct mm_struct *mm)
might_sleep();
if (atomic_dec_and_test(&mm->mm_users)) {
uprobe_clear_state(mm);
exit_aio(mm);
ksm_exit(mm);
khugepaged_exit(mm); /* must run before exit_mmap */
@ -777,6 +782,8 @@ void mm_release(struct task_struct *tsk, struct mm_struct *mm)
exit_pi_state_list(tsk);
#endif
uprobe_free_utask(tsk);
/* Get rid of any cached register state */
deactivate_mm(tsk, mm);
@ -831,6 +838,7 @@ struct mm_struct *dup_mm(struct task_struct *tsk)
#ifdef CONFIG_TRANSPARENT_HUGEPAGE
mm->pmd_huge_pte = NULL;
#endif
uprobe_reset_state(mm);
if (!mm_init(mm, tsk))
goto fail_nomem;
@ -1373,6 +1381,7 @@ static struct task_struct *copy_process(unsigned long clone_flags,
INIT_LIST_HEAD(&p->pi_state_list);
p->pi_state_cache = NULL;
#endif
uprobe_copy_process(p);
/*
* sigaltstack should be cleared when sharing the same VM
*/

4
kernel/signal.c
View File

@ -29,6 +29,7 @@
#include <linux/pid_namespace.h>
#include <linux/nsproxy.h>
#include <linux/user_namespace.h>
#include <linux/uprobes.h>
#define CREATE_TRACE_POINTS
#include <trace/events/signal.h>
@ -2191,6 +2192,9 @@ int get_signal_to_deliver(siginfo_t *info, struct k_sigaction *return_ka,
struct signal_struct *signal = current->signal;
int signr;
if (unlikely(uprobe_deny_signal()))
return 0;
relock:
/*
* We'll jump back here after any time we were stopped in TASK_STOPPED.

20
kernel/trace/Kconfig
View File

@ -372,6 +372,7 @@ config KPROBE_EVENT
depends on HAVE_REGS_AND_STACK_ACCESS_API
bool "Enable kprobes-based dynamic events"
select TRACING
select PROBE_EVENTS
default y
help
This allows the user to add tracing events (similar to tracepoints)
@ -384,6 +385,25 @@ config KPROBE_EVENT
This option is also required by perf-probe subcommand of perf tools.
If you want to use perf tools, this option is strongly recommended.
config UPROBE_EVENT
bool "Enable uprobes-based dynamic events"
depends on ARCH_SUPPORTS_UPROBES
depends on MMU
select UPROBES
select PROBE_EVENTS
select TRACING
default n
help
This allows the user to add tracing events on top of userspace
dynamic events (similar to tracepoints) on the fly via the trace
events interface. Those events can be inserted wherever uprobes
can probe, and record various registers.
This option is required if you plan to use perf-probe subcommand
of perf tools on user space applications.
config PROBE_EVENTS
def_bool n
config DYNAMIC_FTRACE
bool "enable/disable ftrace tracepoints dynamically"
depends on FUNCTION_TRACER

2
kernel/trace/Makefile
View File

@ -60,5 +60,7 @@ endif
ifeq ($(CONFIG_TRACING),y)
obj-$(CONFIG_KGDB_KDB) += trace_kdb.o
endif
obj-$(CONFIG_PROBE_EVENTS) += trace_probe.o
obj-$(CONFIG_UPROBE_EVENT) += trace_uprobe.o
libftrace-y := ftrace.o

5
kernel/trace/trace.h
View File

@ -103,6 +103,11 @@ struct kretprobe_trace_entry_head {
unsigned long ret_ip;
};
struct uprobe_trace_entry_head {
struct trace_entry ent;
unsigned long ip;
};
/*
* trace_flag_type is an enumeration that holds different
* states when a trace occurs. These are:

897
kernel/trace/trace_kprobe.c
View File

File diff suppressed because it is too large Load Diff

839
kernel/trace/trace_probe.c Normal file
View File

@ -0,0 +1,839 @@
/*
* Common code for probe-based Dynamic events.
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License version 2 as
* published by the Free Software Foundation.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
*
* This code was copied from kernel/trace/trace_kprobe.c written by
* Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
*
* Updates to make this generic:
* Copyright (C) IBM Corporation, 2010-2011
* Author: Srikar Dronamraju
*/
#include "trace_probe.h"
const char *reserved_field_names[] = {
"common_type",
"common_flags",
"common_preempt_count",
"common_pid",
"common_tgid",
FIELD_STRING_IP,
FIELD_STRING_RETIP,
FIELD_STRING_FUNC,
};
/* Printing function type */
#define PRINT_TYPE_FUNC_NAME(type) print_type_##type
#define PRINT_TYPE_FMT_NAME(type) print_type_format_##type
/* Printing in basic type function template */
#define DEFINE_BASIC_PRINT_TYPE_FUNC(type, fmt, cast) \
static __kprobes int PRINT_TYPE_FUNC_NAME(type)(struct trace_seq *s, \
const char *name, \
void *data, void *ent)\
{ \
return trace_seq_printf(s, " %s=" fmt, name, (cast)*(type *)data);\
} \
static const char PRINT_TYPE_FMT_NAME(type)[] = fmt;
DEFINE_BASIC_PRINT_TYPE_FUNC(u8, "%x", unsigned int)
DEFINE_BASIC_PRINT_TYPE_FUNC(u16, "%x", unsigned int)
DEFINE_BASIC_PRINT_TYPE_FUNC(u32, "%lx", unsigned long)
DEFINE_BASIC_PRINT_TYPE_FUNC(u64, "%llx", unsigned long long)
DEFINE_BASIC_PRINT_TYPE_FUNC(s8, "%d", int)
DEFINE_BASIC_PRINT_TYPE_FUNC(s16, "%d", int)
DEFINE_BASIC_PRINT_TYPE_FUNC(s32, "%ld", long)
DEFINE_BASIC_PRINT_TYPE_FUNC(s64, "%lld", long long)
static inline void *get_rloc_data(u32 *dl)
{
return (u8 *)dl + get_rloc_offs(*dl);
}
/* For data_loc conversion */
static inline void *get_loc_data(u32 *dl, void *ent)
{
return (u8 *)ent + get_rloc_offs(*dl);
}
/* For defining macros, define string/string_size types */
typedef u32 string;
typedef u32 string_size;
/* Print type function for string type */
static __kprobes int PRINT_TYPE_FUNC_NAME(string)(struct trace_seq *s,
const char *name,
void *data, void *ent)
{
int len = *(u32 *)data >> 16;
if (!len)
return trace_seq_printf(s, " %s=(fault)", name);
else
return trace_seq_printf(s, " %s=\"%s\"", name,
(const char *)get_loc_data(data, ent));
}
static const char PRINT_TYPE_FMT_NAME(string)[] = "\\\"%s\\\"";
#define FETCH_FUNC_NAME(method, type) fetch_##method##_##type
/*
* Define macro for basic types - we don't need to define s* types, because
* we have to care only about bitwidth at recording time.
*/
#define DEFINE_BASIC_FETCH_FUNCS(method) \
DEFINE_FETCH_##method(u8) \
DEFINE_FETCH_##method(u16) \
DEFINE_FETCH_##method(u32) \
DEFINE_FETCH_##method(u64)
#define CHECK_FETCH_FUNCS(method, fn) \
(((FETCH_FUNC_NAME(method, u8) == fn) || \
(FETCH_FUNC_NAME(method, u16) == fn) || \
(FETCH_FUNC_NAME(method, u32) == fn) || \
(FETCH_FUNC_NAME(method, u64) == fn) || \
(FETCH_FUNC_NAME(method, string) == fn) || \
(FETCH_FUNC_NAME(method, string_size) == fn)) \
&& (fn != NULL))
/* Data fetch function templates */
#define DEFINE_FETCH_reg(type) \
static __kprobes void FETCH_FUNC_NAME(reg, type)(struct pt_regs *regs, \
void *offset, void *dest) \
{ \
*(type *)dest = (type)regs_get_register(regs, \
(unsigned int)((unsigned long)offset)); \
}
DEFINE_BASIC_FETCH_FUNCS(reg)
/* No string on the register */
#define fetch_reg_string NULL
#define fetch_reg_string_size NULL
#define DEFINE_FETCH_stack(type) \
static __kprobes void FETCH_FUNC_NAME(stack, type)(struct pt_regs *regs,\
void *offset, void *dest) \
{ \
*(type *)dest = (type)regs_get_kernel_stack_nth(regs, \
(unsigned int)((unsigned long)offset)); \
}
DEFINE_BASIC_FETCH_FUNCS(stack)
/* No string on the stack entry */
#define fetch_stack_string NULL
#define fetch_stack_string_size NULL
#define DEFINE_FETCH_retval(type) \
static __kprobes void FETCH_FUNC_NAME(retval, type)(struct pt_regs *regs,\
void *dummy, void *dest) \
{ \
*(type *)dest = (type)regs_return_value(regs); \
}
DEFINE_BASIC_FETCH_FUNCS(retval)
/* No string on the retval */
#define fetch_retval_string NULL
#define fetch_retval_string_size NULL
#define DEFINE_FETCH_memory(type) \
static __kprobes void FETCH_FUNC_NAME(memory, type)(struct pt_regs *regs,\
void *addr, void *dest) \
{ \
type retval; \
if (probe_kernel_address(addr, retval)) \
*(type *)dest = 0; \
else \
*(type *)dest = retval; \
}
DEFINE_BASIC_FETCH_FUNCS(memory)
/*
* Fetch a null-terminated string. Caller MUST set *(u32 *)dest with max
* length and relative data location.
*/
static __kprobes void FETCH_FUNC_NAME(memory, string)(struct pt_regs *regs,
void *addr, void *dest)
{
long ret;
int maxlen = get_rloc_len(*(u32 *)dest);
u8 *dst = get_rloc_data(dest);
u8 *src = addr;
mm_segment_t old_fs = get_fs();
if (!maxlen)
return;
/*
* Try to get string again, since the string can be changed while
* probing.
*/
set_fs(KERNEL_DS);
pagefault_disable();
do
ret = __copy_from_user_inatomic(dst++, src++, 1);
while (dst[-1] && ret == 0 && src - (u8 *)addr < maxlen);
dst[-1] = '\0';
pagefault_enable();
set_fs(old_fs);
if (ret < 0) { /* Failed to fetch string */
((u8 *)get_rloc_data(dest))[0] = '\0';
*(u32 *)dest = make_data_rloc(0, get_rloc_offs(*(u32 *)dest));
} else {
*(u32 *)dest = make_data_rloc(src - (u8 *)addr,
get_rloc_offs(*(u32 *)dest));
}
}
/* Return the length of string -- including null terminal byte */
static __kprobes void FETCH_FUNC_NAME(memory, string_size)(struct pt_regs *regs,
void *addr, void *dest)
{
mm_segment_t old_fs;
int ret, len = 0;
u8 c;
old_fs = get_fs();
set_fs(KERNEL_DS);
pagefault_disable();
do {
ret = __copy_from_user_inatomic(&c, (u8 *)addr + len, 1);
len++;
} while (c && ret == 0 && len < MAX_STRING_SIZE);
pagefault_enable();
set_fs(old_fs);
if (ret < 0) /* Failed to check the length */
*(u32 *)dest = 0;
else
*(u32 *)dest = len;
}
/* Memory fetching by symbol */
struct symbol_cache {
char *symbol;
long offset;
unsigned long addr;
};
static unsigned long update_symbol_cache(struct symbol_cache *sc)
{
sc->addr = (unsigned long)kallsyms_lookup_name(sc->symbol);
if (sc->addr)
sc->addr += sc->offset;
return sc->addr;
}
static void free_symbol_cache(struct symbol_cache *sc)
{
kfree(sc->symbol);
kfree(sc);
}
static struct symbol_cache *alloc_symbol_cache(const char *sym, long offset)
{
struct symbol_cache *sc;
if (!sym || strlen(sym) == 0)
return NULL;
sc = kzalloc(sizeof(struct symbol_cache), GFP_KERNEL);
if (!sc)
return NULL;
sc->symbol = kstrdup(sym, GFP_KERNEL);
if (!sc->symbol) {
kfree(sc);
return NULL;
}
sc->offset = offset;
update_symbol_cache(sc);
return sc;
}
#define DEFINE_FETCH_symbol(type) \
static __kprobes void FETCH_FUNC_NAME(symbol, type)(struct pt_regs *regs,\
void *data, void *dest) \
{ \
struct symbol_cache *sc = data; \
if (sc->addr) \
fetch_memory_##type(regs, (void *)sc->addr, dest); \
else \
*(type *)dest = 0; \
}
DEFINE_BASIC_FETCH_FUNCS(symbol)
DEFINE_FETCH_symbol(string)
DEFINE_FETCH_symbol(string_size)
/* Dereference memory access function */
struct deref_fetch_param {
struct fetch_param orig;
long offset;
};
#define DEFINE_FETCH_deref(type) \
static __kprobes void FETCH_FUNC_NAME(deref, type)(struct pt_regs *regs,\
void *data, void *dest) \
{ \
struct deref_fetch_param *dprm = data; \
unsigned long addr; \
call_fetch(&dprm->orig, regs, &addr); \
if (addr) { \
addr += dprm->offset; \
fetch_memory_##type(regs, (void *)addr, dest); \
} else \
*(type *)dest = 0; \
}
DEFINE_BASIC_FETCH_FUNCS(deref)
DEFINE_FETCH_deref(string)
DEFINE_FETCH_deref(string_size)
static __kprobes void update_deref_fetch_param(struct deref_fetch_param *data)
{
if (CHECK_FETCH_FUNCS(deref, data->orig.fn))
update_deref_fetch_param(data->orig.data);
else if (CHECK_FETCH_FUNCS(symbol, data->orig.fn))
update_symbol_cache(data->orig.data);
}
static __kprobes void free_deref_fetch_param(struct deref_fetch_param *data)
{
if (CHECK_FETCH_FUNCS(deref, data->orig.fn))
free_deref_fetch_param(data->orig.data);
else if (CHECK_FETCH_FUNCS(symbol, data->orig.fn))
free_symbol_cache(data->orig.data);
kfree(data);
}
/* Bitfield fetch function */
struct bitfield_fetch_param {
struct fetch_param orig;
unsigned char hi_shift;
unsigned char low_shift;
};
#define DEFINE_FETCH_bitfield(type) \
static __kprobes void FETCH_FUNC_NAME(bitfield, type)(struct pt_regs *regs,\
void *data, void *dest) \
{ \
struct bitfield_fetch_param *bprm = data; \
type buf = 0; \
call_fetch(&bprm->orig, regs, &buf); \
if (buf) { \
buf <<= bprm->hi_shift; \
buf >>= bprm->low_shift; \
} \
*(type *)dest = buf; \
}
DEFINE_BASIC_FETCH_FUNCS(bitfield)
#define fetch_bitfield_string NULL
#define fetch_bitfield_string_size NULL
static __kprobes void
update_bitfield_fetch_param(struct bitfield_fetch_param *data)
{
/*
* Don't check the bitfield itself, because this must be the
* last fetch function.
*/
if (CHECK_FETCH_FUNCS(deref, data->orig.fn))
update_deref_fetch_param(data->orig.data);
else if (CHECK_FETCH_FUNCS(symbol, data->orig.fn))
update_symbol_cache(data->orig.data);
}
static __kprobes void
free_bitfield_fetch_param(struct bitfield_fetch_param *data)
{
/*
* Don't check the bitfield itself, because this must be the
* last fetch function.
*/
if (CHECK_FETCH_FUNCS(deref, data->orig.fn))
free_deref_fetch_param(data->orig.data);
else if (CHECK_FETCH_FUNCS(symbol, data->orig.fn))
free_symbol_cache(data->orig.data);
kfree(data);
}
/* Default (unsigned long) fetch type */
#define __DEFAULT_FETCH_TYPE(t) u##t
#define _DEFAULT_FETCH_TYPE(t) __DEFAULT_FETCH_TYPE(t)
#define DEFAULT_FETCH_TYPE _DEFAULT_FETCH_TYPE(BITS_PER_LONG)
#define DEFAULT_FETCH_TYPE_STR __stringify(DEFAULT_FETCH_TYPE)
#define ASSIGN_FETCH_FUNC(method, type) \
[FETCH_MTD_##method] = FETCH_FUNC_NAME(method, type)
#define __ASSIGN_FETCH_TYPE(_name, ptype, ftype, _size, sign, _fmttype) \
{.name = _name, \
.size = _size, \
.is_signed = sign, \
.print = PRINT_TYPE_FUNC_NAME(ptype), \
.fmt = PRINT_TYPE_FMT_NAME(ptype), \
.fmttype = _fmttype, \
.fetch = { \
ASSIGN_FETCH_FUNC(reg, ftype), \
ASSIGN_FETCH_FUNC(stack, ftype), \
ASSIGN_FETCH_FUNC(retval, ftype), \
ASSIGN_FETCH_FUNC(memory, ftype), \
ASSIGN_FETCH_FUNC(symbol, ftype), \
ASSIGN_FETCH_FUNC(deref, ftype), \
ASSIGN_FETCH_FUNC(bitfield, ftype), \
} \
}
#define ASSIGN_FETCH_TYPE(ptype, ftype, sign) \
__ASSIGN_FETCH_TYPE(#ptype, ptype, ftype, sizeof(ftype), sign, #ptype)
#define FETCH_TYPE_STRING 0
#define FETCH_TYPE_STRSIZE 1
/* Fetch type information table */
static const struct fetch_type fetch_type_table[] = {
/* Special types */
[FETCH_TYPE_STRING] = __ASSIGN_FETCH_TYPE("string", string, string,
sizeof(u32), 1, "__data_loc char[]"),
[FETCH_TYPE_STRSIZE] = __ASSIGN_FETCH_TYPE("string_size", u32,
string_size, sizeof(u32), 0, "u32"),
/* Basic types */
ASSIGN_FETCH_TYPE(u8, u8, 0),
ASSIGN_FETCH_TYPE(u16, u16, 0),
ASSIGN_FETCH_TYPE(u32, u32, 0),
ASSIGN_FETCH_TYPE(u64, u64, 0),
ASSIGN_FETCH_TYPE(s8, u8, 1),
ASSIGN_FETCH_TYPE(s16, u16, 1),
ASSIGN_FETCH_TYPE(s32, u32, 1),
ASSIGN_FETCH_TYPE(s64, u64, 1),
};
static const struct fetch_type *find_fetch_type(const char *type)
{
int i;
if (!type)
type = DEFAULT_FETCH_TYPE_STR;
/* Special case: bitfield */
if (*type == 'b') {
unsigned long bs;
type = strchr(type, '/');
if (!type)
goto fail;
type++;
if (strict_strtoul(type, 0, &bs))
goto fail;
switch (bs) {
case 8:
return find_fetch_type("u8");
case 16:
return find_fetch_type("u16");
case 32:
return find_fetch_type("u32");
case 64:
return find_fetch_type("u64");
default:
goto fail;
}
}
for (i = 0; i < ARRAY_SIZE(fetch_type_table); i++)
if (strcmp(type, fetch_type_table[i].name) == 0)
return &fetch_type_table[i];
fail:
return NULL;
}
/* Special function : only accept unsigned long */
static __kprobes void fetch_stack_address(struct pt_regs *regs,
void *dummy, void *dest)
{
*(unsigned long *)dest = kernel_stack_pointer(regs);
}
static fetch_func_t get_fetch_size_function(const struct fetch_type *type,
fetch_func_t orig_fn)
{
int i;
if (type != &fetch_type_table[FETCH_TYPE_STRING])
return NULL; /* Only string type needs size function */
for (i = 0; i < FETCH_MTD_END; i++)
if (type->fetch[i] == orig_fn)
return fetch_type_table[FETCH_TYPE_STRSIZE].fetch[i];
WARN_ON(1); /* This should not happen */
return NULL;
}
/* Split symbol and offset. */
int traceprobe_split_symbol_offset(char *symbol, unsigned long *offset)
{
char *tmp;
int ret;
if (!offset)
return -EINVAL;
tmp = strchr(symbol, '+');
if (tmp) {
/* skip sign because strict_strtol doesn't accept '+' */
ret = strict_strtoul(tmp + 1, 0, offset);
if (ret)
return ret;
*tmp = '\0';
} else
*offset = 0;
return 0;
}
#define PARAM_MAX_STACK (THREAD_SIZE / sizeof(unsigned long))
static int parse_probe_vars(char *arg, const struct fetch_type *t,
struct fetch_param *f, bool is_return)
{
int ret = 0;
unsigned long param;
if (strcmp(arg, "retval") == 0) {
if (is_return)
f->fn = t->fetch[FETCH_MTD_retval];
else
ret = -EINVAL;
} else if (strncmp(arg, "stack", 5) == 0) {
if (arg[5] == '\0') {
if (strcmp(t->name, DEFAULT_FETCH_TYPE_STR) == 0)
f->fn = fetch_stack_address;
else
ret = -EINVAL;
} else if (isdigit(arg[5])) {
ret = strict_strtoul(arg + 5, 10, &param);
if (ret || param > PARAM_MAX_STACK)
ret = -EINVAL;
else {
f->fn = t->fetch[FETCH_MTD_stack];
f->data = (void *)param;
}
} else
ret = -EINVAL;
} else
ret = -EINVAL;
return ret;
}
/* Recursive argument parser */
static int parse_probe_arg(char *arg, const struct fetch_type *t,
struct fetch_param *f, bool is_return, bool is_kprobe)
{
unsigned long param;
long offset;
char *tmp;
int ret;
ret = 0;
/* Until uprobe_events supports only reg arguments */
if (!is_kprobe && arg[0] != '%')
return -EINVAL;
switch (arg[0]) {
case '$':
ret = parse_probe_vars(arg + 1, t, f, is_return);
break;
case '%': /* named register */
ret = regs_query_register_offset(arg + 1);
if (ret >= 0) {
f->fn = t->fetch[FETCH_MTD_reg];
f->data = (void *)(unsigned long)ret;
ret = 0;
}
break;
case '@': /* memory or symbol */
if (isdigit(arg[1])) {
ret = strict_strtoul(arg + 1, 0, &param);
if (ret)
break;
f->fn = t->fetch[FETCH_MTD_memory];
f->data = (void *)param;
} else {
ret = traceprobe_split_symbol_offset(arg + 1, &offset);
if (ret)
break;
f->data = alloc_symbol_cache(arg + 1, offset);
if (f->data)
f->fn = t->fetch[FETCH_MTD_symbol];
}
break;
case '+': /* deref memory */
arg++; /* Skip '+', because strict_strtol() rejects it. */
case '-':
tmp = strchr(arg, '(');
if (!tmp)
break;
*tmp = '\0';
ret = strict_strtol(arg, 0, &offset);
if (ret)
break;
arg = tmp + 1;
tmp = strrchr(arg, ')');
if (tmp) {
struct deref_fetch_param *dprm;
const struct fetch_type *t2;
t2 = find_fetch_type(NULL);
*tmp = '\0';
dprm = kzalloc(sizeof(struct deref_fetch_param), GFP_KERNEL);
if (!dprm)
return -ENOMEM;
dprm->offset = offset;
ret = parse_probe_arg(arg, t2, &dprm->orig, is_return,
is_kprobe);
if (ret)
kfree(dprm);
else {
f->fn = t->fetch[FETCH_MTD_deref];
f->data = (void *)dprm;
}
}
break;
}
if (!ret && !f->fn) { /* Parsed, but do not find fetch method */
pr_info("%s type has no corresponding fetch method.\n", t->name);
ret = -EINVAL;
}
return ret;
}
#define BYTES_TO_BITS(nb) ((BITS_PER_LONG * (nb)) / sizeof(long))
/* Bitfield type needs to be parsed into a fetch function */
static int __parse_bitfield_probe_arg(const char *bf,
const struct fetch_type *t,
struct fetch_param *f)
{
struct bitfield_fetch_param *bprm;
unsigned long bw, bo;
char *tail;
if (*bf != 'b')
return 0;
bprm = kzalloc(sizeof(*bprm), GFP_KERNEL);
if (!bprm)
return -ENOMEM;
bprm->orig = *f;
f->fn = t->fetch[FETCH_MTD_bitfield];
f->data = (void *)bprm;
bw = simple_strtoul(bf + 1, &tail, 0); /* Use simple one */
if (bw == 0 || *tail != '@')
return -EINVAL;
bf = tail + 1;
bo = simple_strtoul(bf, &tail, 0);
if (tail == bf || *tail != '/')
return -EINVAL;
bprm->hi_shift = BYTES_TO_BITS(t->size) - (bw + bo);
bprm->low_shift = bprm->hi_shift + bo;
return (BYTES_TO_BITS(t->size) < (bw + bo)) ? -EINVAL : 0;
}
/* String length checking wrapper */
int traceprobe_parse_probe_arg(char *arg, ssize_t *size,
struct probe_arg *parg, bool is_return, bool is_kprobe)
{
const char *t;
int ret;
if (strlen(arg) > MAX_ARGSTR_LEN) {
pr_info("Argument is too long.: %s\n", arg);
return -ENOSPC;
}
parg->comm = kstrdup(arg, GFP_KERNEL);
if (!parg->comm) {
pr_info("Failed to allocate memory for command '%s'.\n", arg);
return -ENOMEM;
}
t = strchr(parg->comm, ':');
if (t) {
arg[t - parg->comm] = '\0';
t++;
}
parg->type = find_fetch_type(t);
if (!parg->type) {
pr_info("Unsupported type: %s\n", t);
return -EINVAL;
}
parg->offset = *size;
*size += parg->type->size;
ret = parse_probe_arg(arg, parg->type, &parg->fetch, is_return, is_kprobe);
if (ret >= 0 && t != NULL)
ret = __parse_bitfield_probe_arg(t, parg->type, &parg->fetch);
if (ret >= 0) {
parg->fetch_size.fn = get_fetch_size_function(parg->type,
parg->fetch.fn);
parg->fetch_size.data = parg->fetch.data;
}
return ret;
}
/* Return 1 if name is reserved or already used by another argument */
int traceprobe_conflict_field_name(const char *name,
struct probe_arg *args, int narg)
{
int i;
for (i = 0; i < ARRAY_SIZE(reserved_field_names); i++)
if (strcmp(reserved_field_names[i], name) == 0)
return 1;
for (i = 0; i < narg; i++)
if (strcmp(args[i].name, name) == 0)
return 1;
return 0;
}
void traceprobe_update_arg(struct probe_arg *arg)
{
if (CHECK_FETCH_FUNCS(bitfield, arg->fetch.fn))
update_bitfield_fetch_param(arg->fetch.data);
else if (CHECK_FETCH_FUNCS(deref, arg->fetch.fn))
update_deref_fetch_param(arg->fetch.data);
else if (CHECK_FETCH_FUNCS(symbol, arg->fetch.fn))
update_symbol_cache(arg->fetch.data);
}
void traceprobe_free_probe_arg(struct probe_arg *arg)
{
if (CHECK_FETCH_FUNCS(bitfield, arg->fetch.fn))
free_bitfield_fetch_param(arg->fetch.data);
else if (CHECK_FETCH_FUNCS(deref, arg->fetch.fn))
free_deref_fetch_param(arg->fetch.data);
else if (CHECK_FETCH_FUNCS(symbol, arg->fetch.fn))
free_symbol_cache(arg->fetch.data);
kfree(arg->name);
kfree(arg->comm);
}
int traceprobe_command(const char *buf, int (*createfn)(int, char **))
{
char **argv;
int argc, ret;
argc = 0;
ret = 0;
argv = argv_split(GFP_KERNEL, buf, &argc);
if (!argv)
return -ENOMEM;
if (argc)
ret = createfn(argc, argv);
argv_free(argv);
return ret;
}
#define WRITE_BUFSIZE 4096
ssize_t traceprobe_probes_write(struct file *file, const char __user *buffer,
size_t count, loff_t *ppos,
int (*createfn)(int, char **))
{
char *kbuf, *tmp;
int ret = 0;
size_t done = 0;
size_t size;
kbuf = kmalloc(WRITE_BUFSIZE, GFP_KERNEL);
if (!kbuf)
return -ENOMEM;
while (done < count) {
size = count - done;
if (size >= WRITE_BUFSIZE)
size = WRITE_BUFSIZE - 1;
if (copy_from_user(kbuf, buffer + done, size)) {
ret = -EFAULT;
goto out;
}
kbuf[size] = '\0';
tmp = strchr(kbuf, '\n');
if (tmp) {
*tmp = '\0';
size = tmp - kbuf + 1;
} else if (done + size < count) {
pr_warning("Line length is too long: "
"Should be less than %d.", WRITE_BUFSIZE);
ret = -EINVAL;
goto out;
}
done += size;
/* Remove comments */
tmp = strchr(kbuf, '#');
if (tmp)
*tmp = '\0';
ret = traceprobe_command(kbuf, createfn);
if (ret)
goto out;
}
ret = done;
out:
kfree(kbuf);
return ret;
}

161
kernel/trace/trace_probe.h Normal file
View File

@ -0,0 +1,161 @@
/*
* Common header file for probe-based Dynamic events.
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License version 2 as
* published by the Free Software Foundation.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
*
* This code was copied from kernel/trace/trace_kprobe.h written by
* Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
*
* Updates to make this generic:
* Copyright (C) IBM Corporation, 2010-2011
* Author: Srikar Dronamraju
*/
#include <linux/seq_file.h>
#include <linux/slab.h>
#include <linux/smp.h>
#include <linux/debugfs.h>
#include <linux/types.h>
#include <linux/string.h>
#include <linux/ctype.h>
#include <linux/ptrace.h>
#include <linux/perf_event.h>
#include <linux/kprobes.h>
#include <linux/stringify.h>
#include <linux/limits.h>
#include <linux/uaccess.h>
#include <asm/bitsperlong.h>
#include "trace.h"
#include "trace_output.h"
#define MAX_TRACE_ARGS 128
#define MAX_ARGSTR_LEN 63
#define MAX_EVENT_NAME_LEN 64
#define MAX_STRING_SIZE PATH_MAX
/* Reserved field names */
#define FIELD_STRING_IP "__probe_ip"
#define FIELD_STRING_RETIP "__probe_ret_ip"
#define FIELD_STRING_FUNC "__probe_func"
#undef DEFINE_FIELD
#define DEFINE_FIELD(type, item, name, is_signed) \
do { \
ret = trace_define_field(event_call, #type, name, \
offsetof(typeof(field), item), \
sizeof(field.item), is_signed, \
FILTER_OTHER); \
if (ret) \
return ret; \
} while (0)
/* Flags for trace_probe */
#define TP_FLAG_TRACE 1
#define TP_FLAG_PROFILE 2
#define TP_FLAG_REGISTERED 4
#define TP_FLAG_UPROBE 8
/* data_rloc: data relative location, compatible with u32 */
#define make_data_rloc(len, roffs) \
(((u32)(len) << 16) | ((u32)(roffs) & 0xffff))
#define get_rloc_len(dl) ((u32)(dl) >> 16)
#define get_rloc_offs(dl) ((u32)(dl) & 0xffff)
/*
* Convert data_rloc to data_loc:
* data_rloc stores the offset from data_rloc itself, but data_loc
* stores the offset from event entry.
*/
#define convert_rloc_to_loc(dl, offs) ((u32)(dl) + (offs))
/* Data fetch function type */
typedef void (*fetch_func_t)(struct pt_regs *, void *, void *);
/* Printing function type */
typedef int (*print_type_func_t)(struct trace_seq *, const char *, void *, void *);
/* Fetch types */
enum {
FETCH_MTD_reg = 0,
FETCH_MTD_stack,
FETCH_MTD_retval,
FETCH_MTD_memory,
FETCH_MTD_symbol,
FETCH_MTD_deref,
FETCH_MTD_bitfield,
FETCH_MTD_END,
};
/* Fetch type information table */
struct fetch_type {
const char *name; /* Name of type */
size_t size; /* Byte size of type */
int is_signed; /* Signed flag */
print_type_func_t print; /* Print functions */
const char *fmt; /* Fromat string */
const char *fmttype; /* Name in format file */
/* Fetch functions */
fetch_func_t fetch[FETCH_MTD_END];
};
struct fetch_param {
fetch_func_t fn;
void *data;
};
struct probe_arg {
struct fetch_param fetch;
struct fetch_param fetch_size;
unsigned int offset; /* Offset from argument entry */
const char *name; /* Name of this argument */
const char *comm; /* Command of this argument */
const struct fetch_type *type; /* Type of this argument */
};
static inline __kprobes void call_fetch(struct fetch_param *fprm,
struct pt_regs *regs, void *dest)
{
return fprm->fn(regs, fprm->data, dest);
}
/* Check the name is good for event/group/fields */
static inline int is_good_name(const char *name)
{
if (!isalpha(*name) && *name != '_')
return 0;
while (*++name != '\0') {
if (!isalpha(*name) && !isdigit(*name) && *name != '_')
return 0;
}
return 1;
}
extern int traceprobe_parse_probe_arg(char *arg, ssize_t *size,
struct probe_arg *parg, bool is_return, bool is_kprobe);
extern int traceprobe_conflict_field_name(const char *name,
struct probe_arg *args, int narg);
extern void traceprobe_update_arg(struct probe_arg *arg);
extern void traceprobe_free_probe_arg(struct probe_arg *arg);
extern int traceprobe_split_symbol_offset(char *symbol, unsigned long *offset);
extern ssize_t traceprobe_probes_write(struct file *file,
const char __user *buffer, size_t count, loff_t *ppos,
int (*createfn)(int, char**));
extern int traceprobe_command(const char *buf, int (*createfn)(int, char**));

788
kernel/trace/trace_uprobe.c Normal file
View File

@ -0,0 +1,788 @@
/*
* uprobes-based tracing events
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License version 2 as
* published by the Free Software Foundation.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
*
* Copyright (C) IBM Corporation, 2010-2012
* Author: Srikar Dronamraju <srikar@linux.vnet.ibm.com>
*/
#include <linux/module.h>
#include <linux/uaccess.h>
#include <linux/uprobes.h>
#include <linux/namei.h>
#include "trace_probe.h"
#define UPROBE_EVENT_SYSTEM "uprobes"
/*
* uprobe event core functions
*/
struct trace_uprobe;
struct uprobe_trace_consumer {
struct uprobe_consumer cons;
struct trace_uprobe *tu;
};
struct trace_uprobe {
struct list_head list;
struct ftrace_event_class class;
struct ftrace_event_call call;
struct uprobe_trace_consumer *consumer;
struct inode *inode;
char *filename;
unsigned long offset;
unsigned long nhit;
unsigned int flags; /* For TP_FLAG_* */
ssize_t size; /* trace entry size */
unsigned int nr_args;
struct probe_arg args[];
};
#define SIZEOF_TRACE_UPROBE(n) \
(offsetof(struct trace_uprobe, args) + \
(sizeof(struct probe_arg) * (n)))
static int register_uprobe_event(struct trace_uprobe *tu);
static void unregister_uprobe_event(struct trace_uprobe *tu);
static DEFINE_MUTEX(uprobe_lock);
static LIST_HEAD(uprobe_list);
static int uprobe_dispatcher(struct uprobe_consumer *con, struct pt_regs *regs);
/*
* Allocate new trace_uprobe and initialize it (including uprobes).
*/
static struct trace_uprobe *
alloc_trace_uprobe(const char *group, const char *event, int nargs)
{
struct trace_uprobe *tu;
if (!event || !is_good_name(event))
return ERR_PTR(-EINVAL);
if (!group || !is_good_name(group))
return ERR_PTR(-EINVAL);
tu = kzalloc(SIZEOF_TRACE_UPROBE(nargs), GFP_KERNEL);
if (!tu)
return ERR_PTR(-ENOMEM);
tu->call.class = &tu->class;
tu->call.name = kstrdup(event, GFP_KERNEL);
if (!tu->call.name)
goto error;
tu->class.system = kstrdup(group, GFP_KERNEL);
if (!tu->class.system)
goto error;
INIT_LIST_HEAD(&tu->list);
return tu;
error:
kfree(tu->call.name);
kfree(tu);
return ERR_PTR(-ENOMEM);
}
static void free_trace_uprobe(struct trace_uprobe *tu)
{
int i;
for (i = 0; i < tu->nr_args; i++)
traceprobe_free_probe_arg(&tu->args[i]);
iput(tu->inode);
kfree(tu->call.class->system);
kfree(tu->call.name);
kfree(tu->filename);
kfree(tu);
}
static struct trace_uprobe *find_probe_event(const char *event, const char *group)
{
struct trace_uprobe *tu;
list_for_each_entry(tu, &uprobe_list, list)
if (strcmp(tu->call.name, event) == 0 &&
strcmp(tu->call.class->system, group) == 0)
return tu;
return NULL;
}
/* Unregister a trace_uprobe and probe_event: call with locking uprobe_lock */
static void unregister_trace_uprobe(struct trace_uprobe *tu)
{
list_del(&tu->list);
unregister_uprobe_event(tu);
free_trace_uprobe(tu);
}
/* Register a trace_uprobe and probe_event */
static int register_trace_uprobe(struct trace_uprobe *tu)
{
struct trace_uprobe *old_tp;
int ret;
mutex_lock(&uprobe_lock);
/* register as an event */
old_tp = find_probe_event(tu->call.name, tu->call.class->system);
if (old_tp)
/* delete old event */
unregister_trace_uprobe(old_tp);
ret = register_uprobe_event(tu);
if (ret) {
pr_warning("Failed to register probe event(%d)\n", ret);
goto end;
}
list_add_tail(&tu->list, &uprobe_list);
end:
mutex_unlock(&uprobe_lock);
return ret;
}
/*
* Argument syntax:
* - Add uprobe: p[:[GRP/]EVENT] PATH:SYMBOL[+offs] [FETCHARGS]
*
* - Remove uprobe: -:[GRP/]EVENT
*/
static int create_trace_uprobe(int argc, char **argv)
{
struct trace_uprobe *tu;
struct inode *inode;
char *arg, *event, *group, *filename;
char buf[MAX_EVENT_NAME_LEN];
struct path path;
unsigned long offset;
bool is_delete;
int i, ret;
inode = NULL;
ret = 0;
is_delete = false;
event = NULL;
group = NULL;
/* argc must be >= 1 */
if (argv[0][0] == '-')
is_delete = true;
else if (argv[0][0] != 'p') {
pr_info("Probe definition must be started with 'p', 'r' or" " '-'.\n");
return -EINVAL;
}
if (argv[0][1] == ':') {
event = &argv[0][2];
arg = strchr(event, '/');
if (arg) {
group = event;
event = arg + 1;
event[-1] = '\0';
if (strlen(group) == 0) {
pr_info("Group name is not specified\n");
return -EINVAL;
}
}
if (strlen(event) == 0) {
pr_info("Event name is not specified\n");
return -EINVAL;
}
}
if (!group)
group = UPROBE_EVENT_SYSTEM;
if (is_delete) {
if (!event) {
pr_info("Delete command needs an event name.\n");
return -EINVAL;
}
mutex_lock(&uprobe_lock);
tu = find_probe_event(event, group);
if (!tu) {
mutex_unlock(&uprobe_lock);
pr_info("Event %s/%s doesn't exist.\n", group, event);
return -ENOENT;
}
/* delete an event */
unregister_trace_uprobe(tu);
mutex_unlock(&uprobe_lock);
return 0;
}
if (argc < 2) {
pr_info("Probe point is not specified.\n");
return -EINVAL;
}
if (isdigit(argv[1][0])) {
pr_info("probe point must be have a filename.\n");
return -EINVAL;
}
arg = strchr(argv[1], ':');
if (!arg)
goto fail_address_parse;
*arg++ = '\0';
filename = argv[1];
ret = kern_path(filename, LOOKUP_FOLLOW, &path);
if (ret)
goto fail_address_parse;
ret = strict_strtoul(arg, 0, &offset);
if (ret)
goto fail_address_parse;
inode = igrab(path.dentry->d_inode);
argc -= 2;
argv += 2;
/* setup a probe */
if (!event) {
char *tail = strrchr(filename, '/');
char *ptr;
ptr = kstrdup((tail ? tail + 1 : filename), GFP_KERNEL);
if (!ptr) {
ret = -ENOMEM;
goto fail_address_parse;
}
tail = ptr;
ptr = strpbrk(tail, ".-_");
if (ptr)
*ptr = '\0';
snprintf(buf, MAX_EVENT_NAME_LEN, "%c_%s_0x%lx", 'p', tail, offset);
event = buf;
kfree(tail);
}
tu = alloc_trace_uprobe(group, event, argc);
if (IS_ERR(tu)) {
pr_info("Failed to allocate trace_uprobe.(%d)\n", (int)PTR_ERR(tu));
ret = PTR_ERR(tu);
goto fail_address_parse;
}
tu->offset = offset;
tu->inode = inode;
tu->filename = kstrdup(filename, GFP_KERNEL);
if (!tu->filename) {
pr_info("Failed to allocate filename.\n");
ret = -ENOMEM;
goto error;
}
/* parse arguments */
ret = 0;
for (i = 0; i < argc && i < MAX_TRACE_ARGS; i++) {
/* Increment count for freeing args in error case */
tu->nr_args++;
/* Parse argument name */
arg = strchr(argv[i], '=');
if (arg) {
*arg++ = '\0';
tu->args[i].name = kstrdup(argv[i], GFP_KERNEL);
} else {
arg = argv[i];
/* If argument name is omitted, set "argN" */
snprintf(buf, MAX_EVENT_NAME_LEN, "arg%d", i + 1);
tu->args[i].name = kstrdup(buf, GFP_KERNEL);
}
if (!tu->args[i].name) {
pr_info("Failed to allocate argument[%d] name.\n", i);
ret = -ENOMEM;
goto error;
}
if (!is_good_name(tu->args[i].name)) {
pr_info("Invalid argument[%d] name: %s\n", i, tu->args[i].name);
ret = -EINVAL;
goto error;
}
if (traceprobe_conflict_field_name(tu->args[i].name, tu->args, i)) {
pr_info("Argument[%d] name '%s' conflicts with "
"another field.\n", i, argv[i]);
ret = -EINVAL;
goto error;
}
/* Parse fetch argument */
ret = traceprobe_parse_probe_arg(arg, &tu->size, &tu->args[i], false, false);
if (ret) {
pr_info("Parse error at argument[%d]. (%d)\n", i, ret);
goto error;
}
}
ret = register_trace_uprobe(tu);
if (ret)
goto error;
return 0;
error:
free_trace_uprobe(tu);
return ret;
fail_address_parse:
if (inode)
iput(inode);
pr_info("Failed to parse address.\n");
return ret;
}
static void cleanup_all_probes(void)
{
struct trace_uprobe *tu;
mutex_lock(&uprobe_lock);
while (!list_empty(&uprobe_list)) {
tu = list_entry(uprobe_list.next, struct trace_uprobe, list);
unregister_trace_uprobe(tu);
}
mutex_unlock(&uprobe_lock);
}
/* Probes listing interfaces */
static void *probes_seq_start(struct seq_file *m, loff_t *pos)
{
mutex_lock(&uprobe_lock);
return seq_list_start(&uprobe_list, *pos);
}
static void *probes_seq_next(struct seq_file *m, void *v, loff_t *pos)
{
return seq_list_next(v, &uprobe_list, pos);
}
static void probes_seq_stop(struct seq_file *m, void *v)
{
mutex_unlock(&uprobe_lock);
}
static int probes_seq_show(struct seq_file *m, void *v)
{
struct trace_uprobe *tu = v;
int i;
seq_printf(m, "p:%s/%s", tu->call.class->system, tu->call.name);
seq_printf(m, " %s:0x%p", tu->filename, (void *)tu->offset);
for (i = 0; i < tu->nr_args; i++)
seq_printf(m, " %s=%s", tu->args[i].name, tu->args[i].comm);
seq_printf(m, "\n");
return 0;
}
static const struct seq_operations probes_seq_op = {
.start = probes_seq_start,
.next = probes_seq_next,
.stop = probes_seq_stop,
.show = probes_seq_show
};
static int probes_open(struct inode *inode, struct file *file)
{
if ((file->f_mode & FMODE_WRITE) && (file->f_flags & O_TRUNC))
cleanup_all_probes();
return seq_open(file, &probes_seq_op);
}
static ssize_t probes_write(struct file *file, const char __user *buffer,
size_t count, loff_t *ppos)
{
return traceprobe_probes_write(file, buffer, count, ppos, create_trace_uprobe);
}
static const struct file_operations uprobe_events_ops = {
.owner = THIS_MODULE,
.open = probes_open,
.read = seq_read,
.llseek = seq_lseek,
.release = seq_release,
.write = probes_write,
};
/* Probes profiling interfaces */
static int probes_profile_seq_show(struct seq_file *m, void *v)
{
struct trace_uprobe *tu = v;
seq_printf(m, " %s %-44s %15lu\n", tu->filename, tu->call.name, tu->nhit);
return 0;
}
static const struct seq_operations profile_seq_op = {
.start = probes_seq_start,
.next = probes_seq_next,
.stop = probes_seq_stop,
.show = probes_profile_seq_show
};
static int profile_open(struct inode *inode, struct file *file)
{
return seq_open(file, &profile_seq_op);
}
static const struct file_operations uprobe_profile_ops = {
.owner = THIS_MODULE,
.open = profile_open,
.read = seq_read,
.llseek = seq_lseek,
.release = seq_release,
};
/* uprobe handler */
static void uprobe_trace_func(struct trace_uprobe *tu, struct pt_regs *regs)
{
struct uprobe_trace_entry_head *entry;
struct ring_buffer_event *event;
struct ring_buffer *buffer;
u8 *data;
int size, i, pc;
unsigned long irq_flags;
struct ftrace_event_call *call = &tu->call;
tu->nhit++;
local_save_flags(irq_flags);
pc = preempt_count();
size = sizeof(*entry) + tu->size;
event = trace_current_buffer_lock_reserve(&buffer, call->event.type,
size, irq_flags, pc);
if (!event)
return;
entry = ring_buffer_event_data(event);
entry->ip = uprobe_get_swbp_addr(task_pt_regs(current));
data = (u8 *)&entry[1];
for (i = 0; i < tu->nr_args; i++)
call_fetch(&tu->args[i].fetch, regs, data + tu->args[i].offset);
if (!filter_current_check_discard(buffer, call, entry, event))
trace_buffer_unlock_commit(buffer, event, irq_flags, pc);
}
/* Event entry printers */
static enum print_line_t
print_uprobe_event(struct trace_iterator *iter, int flags, struct trace_event *event)
{
struct uprobe_trace_entry_head *field;
struct trace_seq *s = &iter->seq;
struct trace_uprobe *tu;
u8 *data;
int i;
field = (struct uprobe_trace_entry_head *)iter->ent;
tu = container_of(event, struct trace_uprobe, call.event);
if (!trace_seq_printf(s, "%s: (", tu->call.name))
goto partial;
if (!seq_print_ip_sym(s, field->ip, flags | TRACE_ITER_SYM_OFFSET))
goto partial;
if (!trace_seq_puts(s, ")"))
goto partial;
data = (u8 *)&field[1];
for (i = 0; i < tu->nr_args; i++) {
if (!tu->args[i].type->print(s, tu->args[i].name,
data + tu->args[i].offset, field))
goto partial;
}
if (trace_seq_puts(s, "\n"))
return TRACE_TYPE_HANDLED;
partial:
return TRACE_TYPE_PARTIAL_LINE;
}
static int probe_event_enable(struct trace_uprobe *tu, int flag)
{
struct uprobe_trace_consumer *utc;
int ret = 0;
if (!tu->inode || tu->consumer)
return -EINTR;
utc = kzalloc(sizeof(struct uprobe_trace_consumer), GFP_KERNEL);
if (!utc)
return -EINTR;
utc->cons.handler = uprobe_dispatcher;
utc->cons.filter = NULL;
ret = uprobe_register(tu->inode, tu->offset, &utc->cons);
if (ret) {
kfree(utc);
return ret;
}
tu->flags |= flag;
utc->tu = tu;
tu->consumer = utc;
return 0;
}
static void probe_event_disable(struct trace_uprobe *tu, int flag)
{
if (!tu->inode || !tu->consumer)
return;
uprobe_unregister(tu->inode, tu->offset, &tu->consumer->cons);
tu->flags &= ~flag;
kfree(tu->consumer);
tu->consumer = NULL;
}
static int uprobe_event_define_fields(struct ftrace_event_call *event_call)
{
int ret, i;
struct uprobe_trace_entry_head field;
struct trace_uprobe *tu = (struct trace_uprobe *)event_call->data;
DEFINE_FIELD(unsigned long, ip, FIELD_STRING_IP, 0);
/* Set argument names as fields */
for (i = 0; i < tu->nr_args; i++) {
ret = trace_define_field(event_call, tu->args[i].type->fmttype,
tu->args[i].name,
sizeof(field) + tu->args[i].offset,
tu->args[i].type->size,
tu->args[i].type->is_signed,
FILTER_OTHER);
if (ret)
return ret;
}
return 0;
}
#define LEN_OR_ZERO (len ? len - pos : 0)
static int __set_print_fmt(struct trace_uprobe *tu, char *buf, int len)
{
const char *fmt, *arg;
int i;
int pos = 0;
fmt = "(%lx)";
arg = "REC->" FIELD_STRING_IP;
/* When len=0, we just calculate the needed length */
pos += snprintf(buf + pos, LEN_OR_ZERO, "\"%s", fmt);
for (i = 0; i < tu->nr_args; i++) {
pos += snprintf(buf + pos, LEN_OR_ZERO, " %s=%s",
tu->args[i].name, tu->args[i].type->fmt);
}
pos += snprintf(buf + pos, LEN_OR_ZERO, "\", %s", arg);
for (i = 0; i < tu->nr_args; i++) {
pos += snprintf(buf + pos, LEN_OR_ZERO, ", REC->%s",
tu->args[i].name);
}
return pos; /* return the length of print_fmt */
}
#undef LEN_OR_ZERO
static int set_print_fmt(struct trace_uprobe *tu)
{
char *print_fmt;
int len;
/* First: called with 0 length to calculate the needed length */
len = __set_print_fmt(tu, NULL, 0);
print_fmt = kmalloc(len + 1, GFP_KERNEL);
if (!print_fmt)
return -ENOMEM;
/* Second: actually write the @print_fmt */
__set_print_fmt(tu, print_fmt, len + 1);
tu->call.print_fmt = print_fmt;
return 0;
}
#ifdef CONFIG_PERF_EVENTS
/* uprobe profile handler */
static void uprobe_perf_func(struct trace_uprobe *tu, struct pt_regs *regs)
{
struct ftrace_event_call *call = &tu->call;
struct uprobe_trace_entry_head *entry;
struct hlist_head *head;
u8 *data;
int size, __size, i;
int rctx;
__size = sizeof(*entry) + tu->size;
size = ALIGN(__size + sizeof(u32), sizeof(u64));
size -= sizeof(u32);
if (WARN_ONCE(size > PERF_MAX_TRACE_SIZE, "profile buffer not large enough"))
return;
preempt_disable();
entry = perf_trace_buf_prepare(size, call->event.type, regs, &rctx);
if (!entry)
goto out;
entry->ip = uprobe_get_swbp_addr(task_pt_regs(current));
data = (u8 *)&entry[1];
for (i = 0; i < tu->nr_args; i++)
call_fetch(&tu->args[i].fetch, regs, data + tu->args[i].offset);
head = this_cpu_ptr(call->perf_events);
perf_trace_buf_submit(entry, size, rctx, entry->ip, 1, regs, head);
out:
preempt_enable();
}
#endif /* CONFIG_PERF_EVENTS */
static
int trace_uprobe_register(struct ftrace_event_call *event, enum trace_reg type, void *data)
{
struct trace_uprobe *tu = (struct trace_uprobe *)event->data;
switch (type) {
case TRACE_REG_REGISTER:
return probe_event_enable(tu, TP_FLAG_TRACE);
case TRACE_REG_UNREGISTER:
probe_event_disable(tu, TP_FLAG_TRACE);
return 0;
#ifdef CONFIG_PERF_EVENTS
case TRACE_REG_PERF_REGISTER:
return probe_event_enable(tu, TP_FLAG_PROFILE);
case TRACE_REG_PERF_UNREGISTER:
probe_event_disable(tu, TP_FLAG_PROFILE);
return 0;
#endif
default:
return 0;
}
return 0;
}
static int uprobe_dispatcher(struct uprobe_consumer *con, struct pt_regs *regs)
{
struct uprobe_trace_consumer *utc;
struct trace_uprobe *tu;
utc = container_of(con, struct uprobe_trace_consumer, cons);
tu = utc->tu;
if (!tu || tu->consumer != utc)
return 0;
if (tu->flags & TP_FLAG_TRACE)
uprobe_trace_func(tu, regs);
#ifdef CONFIG_PERF_EVENTS
if (tu->flags & TP_FLAG_PROFILE)
uprobe_perf_func(tu, regs);
#endif
return 0;
}
static struct trace_event_functions uprobe_funcs = {
.trace = print_uprobe_event
};
static int register_uprobe_event(struct trace_uprobe *tu)
{
struct ftrace_event_call *call = &tu->call;
int ret;
/* Initialize ftrace_event_call */
INIT_LIST_HEAD(&call->class->fields);
call->event.funcs = &uprobe_funcs;
call->class->define_fields = uprobe_event_define_fields;
if (set_print_fmt(tu) < 0)
return -ENOMEM;
ret = register_ftrace_event(&call->event);
if (!ret) {
kfree(call->print_fmt);
return -ENODEV;
}
call->flags = 0;
call->class->reg = trace_uprobe_register;
call->data = tu;
ret = trace_add_event_call(call);
if (ret) {
pr_info("Failed to register uprobe event: %s\n", call->name);
kfree(call->print_fmt);
unregister_ftrace_event(&call->event);
}
return ret;
}
static void unregister_uprobe_event(struct trace_uprobe *tu)
{
/* tu->event is unregistered in trace_remove_event_call() */
trace_remove_event_call(&tu->call);
kfree(tu->call.print_fmt);
tu->call.print_fmt = NULL;
}
/* Make a trace interface for controling probe points */
static __init int init_uprobe_trace(void)
{
struct dentry *d_tracer;
d_tracer = tracing_init_dentry();
if (!d_tracer)
return 0;
trace_create_file("uprobe_events", 0644, d_tracer,
NULL, &uprobe_events_ops);
/* Profile interface */
trace_create_file("uprobe_profile", 0444, d_tracer,
NULL, &uprobe_profile_ops);
return 0;
}
fs_initcall(init_uprobe_trace);

3
mm/memory.c
View File

@ -1307,6 +1307,9 @@ static void unmap_single_vma(struct mmu_gather *tlb,
if (end <= vma->vm_start)
return;
if (vma->vm_file)
uprobe_munmap(vma, start, end);
if (unlikely(is_pfn_mapping(vma)))
untrack_pfn_vma(vma, 0, 0);

33
mm/mmap.c
View File

@ -30,6 +30,7 @@
#include <linux/perf_event.h>
#include <linux/audit.h>
#include <linux/khugepaged.h>
#include <linux/uprobes.h>
#include <asm/uaccess.h>
#include <asm/cacheflush.h>
@ -546,8 +547,15 @@ again: remove_next = 1 + (end > next->vm_end);
if (file) {
mapping = file->f_mapping;
if (!(vma->vm_flags & VM_NONLINEAR))
if (!(vma->vm_flags & VM_NONLINEAR)) {
root = &mapping->i_mmap;
uprobe_munmap(vma, vma->vm_start, vma->vm_end);
if (adjust_next)
uprobe_munmap(next, next->vm_start,
next->vm_end);
}
mutex_lock(&mapping->i_mmap_mutex);
if (insert) {
/*
@ -617,8 +625,16 @@ again: remove_next = 1 + (end > next->vm_end);
if (mapping)
mutex_unlock(&mapping->i_mmap_mutex);
if (root) {
uprobe_mmap(vma);
if (adjust_next)
uprobe_mmap(next);
}
if (remove_next) {
if (file) {
uprobe_munmap(next, next->vm_start, next->vm_end);
fput(file);
if (next->vm_flags & VM_EXECUTABLE)
removed_exe_file_vma(mm);
@ -638,6 +654,8 @@ again: remove_next = 1 + (end > next->vm_end);
goto again;
}
}
if (insert && file)
uprobe_mmap(insert);
validate_mm(mm);
@ -1371,6 +1389,11 @@ out:
mm->locked_vm += (len >> PAGE_SHIFT);
} else if ((flags & MAP_POPULATE) && !(flags & MAP_NONBLOCK))
make_pages_present(addr, addr + len);
if (file && uprobe_mmap(vma))
/* matching probes but cannot insert */
goto unmap_and_free_vma;
return addr;
unmap_and_free_vma:
@ -2358,6 +2381,10 @@ int insert_vm_struct(struct mm_struct * mm, struct vm_area_struct * vma)
if ((vma->vm_flags & VM_ACCOUNT) &&
security_vm_enough_memory_mm(mm, vma_pages(vma)))
return -ENOMEM;
if (vma->vm_file && uprobe_mmap(vma))
return -EINVAL;
vma_link(mm, vma, prev, rb_link, rb_parent);
return 0;
}
@ -2427,6 +2454,10 @@ struct vm_area_struct *copy_vma(struct vm_area_struct **vmap,
new_vma->vm_pgoff = pgoff;
if (new_vma->vm_file) {
get_file(new_vma->vm_file);
if (uprobe_mmap(new_vma))
goto out_free_mempol;
if (vma->vm_flags & VM_EXECUTABLE)
added_exe_file_vma(mm);
}

19
tools/perf/Documentation/perf-probe.txt
View File

@ -77,7 +77,8 @@ OPTIONS
-F::
--funcs::
Show available functions in given module or kernel.
Show available functions in given module or kernel. With -x/--exec,
can also list functions in a user space executable / shared library.
--filter=FILTER::
(Only for --vars and --funcs) Set filter. FILTER is a combination of glob
@ -98,6 +99,15 @@ OPTIONS
--max-probes::
Set the maximum number of probe points for an event. Default is 128.
-x::
--exec=PATH::
Specify path to the executable or shared library file for user
space tracing. Can also be used with --funcs option.
In absence of -m/-x options, perf probe checks if the first argument after
the options is an absolute path name. If its an absolute path, perf probe
uses it as a target module/target user space binary to probe.
PROBE SYNTAX
------------
Probe points are defined by following syntax.
@ -182,6 +192,13 @@ Delete all probes on schedule().
./perf probe --del='schedule*'
Add probes at zfree() function on /bin/zsh
./perf probe -x /bin/zsh zfree or ./perf probe /bin/zsh zfree
Add probes at malloc() function on libc
./perf probe -x /lib/libc.so.6 malloc or ./perf probe /lib/libc.so.6 malloc
SEE ALSO
--------

86
tools/perf/builtin-probe.c
View File

@ -54,6 +54,7 @@ static struct {
bool show_ext_vars;
bool show_funcs;
bool mod_events;
bool uprobes;
int nevents;
struct perf_probe_event events[MAX_PROBES];
struct strlist *dellist;
@ -75,6 +76,8 @@ static int parse_probe_event(const char *str)
return -1;
}
pev->uprobes = params.uprobes;
/* Parse a perf-probe command into event */
ret = parse_perf_probe_command(str, pev);
pr_debug("%d arguments\n", pev->nargs);
@ -82,21 +85,58 @@ static int parse_probe_event(const char *str)
return ret;
}
static int set_target(const char *ptr)
{
int found = 0;
const char *buf;
/*
* The first argument after options can be an absolute path
* to an executable / library or kernel module.
*
* TODO: Support relative path, and $PATH, $LD_LIBRARY_PATH,
* short module name.
*/
if (!params.target && ptr && *ptr == '/') {
params.target = ptr;
found = 1;
buf = ptr + (strlen(ptr) - 3);
if (strcmp(buf, ".ko"))
params.uprobes = true;
}
return found;
}
static int parse_probe_event_argv(int argc, const char **argv)
{
int i, len, ret;
int i, len, ret, found_target;
char *buf;
found_target = set_target(argv[0]);
if (found_target && argc == 1)
return 0;
/* Bind up rest arguments */
len = 0;
for (i = 0; i < argc; i++)
for (i = 0; i < argc; i++) {
if (i == 0 && found_target)
continue;
len += strlen(argv[i]) + 1;
}
buf = zalloc(len + 1);
if (buf == NULL)
return -ENOMEM;
len = 0;
for (i = 0; i < argc; i++)
for (i = 0; i < argc; i++) {
if (i == 0 && found_target)
continue;
len += sprintf(&buf[len], "%s ", argv[i]);
}
params.mod_events = true;
ret = parse_probe_event(buf);
free(buf);
@ -125,6 +165,28 @@ static int opt_del_probe_event(const struct option *opt __used,
return 0;
}
static int opt_set_target(const struct option *opt, const char *str,
int unset __used)
{
int ret = -ENOENT;
if (str && !params.target) {
if (!strcmp(opt->long_name, "exec"))
params.uprobes = true;
#ifdef DWARF_SUPPORT
else if (!strcmp(opt->long_name, "module"))
params.uprobes = false;
#endif
else
return ret;
params.target = str;
ret = 0;
}
return ret;
}
#ifdef DWARF_SUPPORT
static int opt_show_lines(const struct option *opt __used,
const char *str, int unset __used)
@ -246,9 +308,9 @@ static const struct option options[] = {
"file", "vmlinux pathname"),
OPT_STRING('s', "source", &symbol_conf.source_prefix,
"directory", "path to kernel source"),
OPT_STRING('m', "module", &params.target,
"modname|path",
"target module name (for online) or path (for offline)"),
OPT_CALLBACK('m', "module", NULL, "modname|path",
"target module name (for online) or path (for offline)",
opt_set_target),
#endif
OPT__DRY_RUN(&probe_event_dry_run),
OPT_INTEGER('\0', "max-probes", &params.max_probe_points,
@ -260,6 +322,8 @@ static const struct option options[] = {
"\t\t\t(default: \"" DEFAULT_VAR_FILTER "\" for --vars,\n"
"\t\t\t \"" DEFAULT_FUNC_FILTER "\" for --funcs)",
opt_set_filter),
OPT_CALLBACK('x', "exec", NULL, "executable|path",
"target executable name or path", opt_set_target),
OPT_END()
};
@ -310,6 +374,10 @@ int cmd_probe(int argc, const char **argv, const char *prefix __used)
pr_err(" Error: Don't use --list with --funcs.\n");
usage_with_options(probe_usage, options);
}
if (params.uprobes) {
pr_warning(" Error: Don't use --list with --exec.\n");
usage_with_options(probe_usage, options);
}
ret = show_perf_probe_events();
if (ret < 0)
pr_err(" Error: Failed to show event list. (%d)\n",
@ -333,8 +401,8 @@ int cmd_probe(int argc, const char **argv, const char *prefix __used)
if (!params.filter)
params.filter = strfilter__new(DEFAULT_FUNC_FILTER,
NULL);
ret = show_available_funcs(params.target,
params.filter);
ret = show_available_funcs(params.target, params.filter,
params.uprobes);
strfilter__delete(params.filter);
if (ret < 0)
pr_err(" Error: Failed to show functions."
@ -343,7 +411,7 @@ int cmd_probe(int argc, const char **argv, const char *prefix __used)
}
#ifdef DWARF_SUPPORT
if (params.show_lines) {
if (params.show_lines && !params.uprobes) {
if (params.mod_events) {
pr_err(" Error: Don't use --line with"
" --add/--del.\n");

434
tools/perf/util/probe-event.c
View File

@ -44,6 +44,7 @@
#include "trace-event.h" /* For __unused */
#include "probe-event.h"
#include "probe-finder.h"
#include "session.h"
#define MAX_CMDLEN 256
#define MAX_PROBE_ARGS 128
@ -70,6 +71,8 @@ static int e_snprintf(char *str, size_t size, const char *format, ...)
}
static char *synthesize_perf_probe_point(struct perf_probe_point *pp);
static int convert_name_to_addr(struct perf_probe_event *pev,
const char *exec);
static struct machine machine;
/* Initialize symbol maps and path of vmlinux/modules */
@ -170,6 +173,34 @@ const char *kernel_get_module_path(const char *module)
return (dso) ? dso->long_name : NULL;
}
static int init_user_exec(void)
{
int ret = 0;
symbol_conf.try_vmlinux_path = false;
symbol_conf.sort_by_name = true;
ret = symbol__init();
if (ret < 0)
pr_debug("Failed to init symbol map.\n");
return ret;
}
static int convert_to_perf_probe_point(struct probe_trace_point *tp,
struct perf_probe_point *pp)
{
pp->function = strdup(tp->symbol);
if (pp->function == NULL)
return -ENOMEM;
pp->offset = tp->offset;
pp->retprobe = tp->retprobe;
return 0;
}
#ifdef DWARF_SUPPORT
/* Open new debuginfo of given module */
static struct debuginfo *open_debuginfo(const char *module)
@ -224,10 +255,7 @@ static int kprobe_convert_to_perf_probe(struct probe_trace_point *tp,
if (ret <= 0) {
pr_debug("Failed to find corresponding probes from "
"debuginfo. Use kprobe event information.\n");
pp->function = strdup(tp->symbol);
if (pp->function == NULL)
return -ENOMEM;
pp->offset = tp->offset;
return convert_to_perf_probe_point(tp, pp);
}
pp->retprobe = tp->retprobe;
@ -275,9 +303,20 @@ static int try_to_find_probe_trace_events(struct perf_probe_event *pev,
int max_tevs, const char *target)
{
bool need_dwarf = perf_probe_event_need_dwarf(pev);
struct debuginfo *dinfo = open_debuginfo(target);
struct debuginfo *dinfo;
int ntevs, ret = 0;
if (pev->uprobes) {
if (need_dwarf) {
pr_warning("Debuginfo-analysis is not yet supported"
" with -x/--exec option.\n");
return -ENOSYS;
}
return convert_name_to_addr(pev, target);
}
dinfo = open_debuginfo(target);
if (!dinfo) {
if (need_dwarf) {
pr_warning("Failed to open debuginfo file.\n");
@ -603,23 +642,22 @@ static int kprobe_convert_to_perf_probe(struct probe_trace_point *tp,
pr_err("Failed to find symbol %s in kernel.\n", tp->symbol);
return -ENOENT;
}
pp->function = strdup(tp->symbol);
if (pp->function == NULL)
return -ENOMEM;
pp->offset = tp->offset;
pp->retprobe = tp->retprobe;
return 0;
return convert_to_perf_probe_point(tp, pp);
}
static int try_to_find_probe_trace_events(struct perf_probe_event *pev,
struct probe_trace_event **tevs __unused,
int max_tevs __unused, const char *mod __unused)
int max_tevs __unused, const char *target)
{
if (perf_probe_event_need_dwarf(pev)) {
pr_warning("Debuginfo-analysis is not supported.\n");
return -ENOSYS;
}
if (pev->uprobes)
return convert_name_to_addr(pev, target);
return 0;
}
@ -1341,11 +1379,18 @@ char *synthesize_probe_trace_command(struct probe_trace_event *tev)
if (buf == NULL)
return NULL;
len = e_snprintf(buf, MAX_CMDLEN, "%c:%s/%s %s%s%s+%lu",
tp->retprobe ? 'r' : 'p',
tev->group, tev->event,
tp->module ?: "", tp->module ? ":" : "",
tp->symbol, tp->offset);
if (tev->uprobes)
len = e_snprintf(buf, MAX_CMDLEN, "%c:%s/%s %s:%s",
tp->retprobe ? 'r' : 'p',
tev->group, tev->event,
tp->module, tp->symbol);
else
len = e_snprintf(buf, MAX_CMDLEN, "%c:%s/%s %s%s%s+%lu",
tp->retprobe ? 'r' : 'p',
tev->group, tev->event,
tp->module ?: "", tp->module ? ":" : "",
tp->symbol, tp->offset);
if (len <= 0)
goto error;
@ -1364,7 +1409,7 @@ error:
}
static int convert_to_perf_probe_event(struct probe_trace_event *tev,
struct perf_probe_event *pev)
struct perf_probe_event *pev, bool is_kprobe)
{
char buf[64] = "";
int i, ret;
@ -1376,7 +1421,11 @@ static int convert_to_perf_probe_event(struct probe_trace_event *tev,
return -ENOMEM;
/* Convert trace_point to probe_point */
ret = kprobe_convert_to_perf_probe(&tev->point, &pev->point);
if (is_kprobe)
ret = kprobe_convert_to_perf_probe(&tev->point, &pev->point);
else
ret = convert_to_perf_probe_point(&tev->point, &pev->point);
if (ret < 0)
return ret;
@ -1472,7 +1521,26 @@ static void clear_probe_trace_event(struct probe_trace_event *tev)
memset(tev, 0, sizeof(*tev));
}
static int open_kprobe_events(bool readwrite)
static void print_warn_msg(const char *file, bool is_kprobe)
{
if (errno == ENOENT) {
const char *config;
if (!is_kprobe)
config = "CONFIG_UPROBE_EVENTS";
else
config = "CONFIG_KPROBE_EVENTS";
pr_warning("%s file does not exist - please rebuild kernel"
" with %s.\n", file, config);
} else
pr_warning("Failed to open %s file: %s\n", file,
strerror(errno));
}
static int open_probe_events(const char *trace_file, bool readwrite,
bool is_kprobe)
{
char buf[PATH_MAX];
const char *__debugfs;
@ -1484,27 +1552,31 @@ static int open_kprobe_events(bool readwrite)
return -ENOENT;
}
ret = e_snprintf(buf, PATH_MAX, "%stracing/kprobe_events", __debugfs);
ret = e_snprintf(buf, PATH_MAX, "%s/%s", __debugfs, trace_file);
if (ret >= 0) {
pr_debug("Opening %s write=%d\n", buf, readwrite);
if (readwrite && !probe_event_dry_run)
ret = open(buf, O_RDWR, O_APPEND);
else
ret = open(buf, O_RDONLY, 0);
}
if (ret < 0) {
if (errno == ENOENT)
pr_warning("kprobe_events file does not exist - please"
" rebuild kernel with CONFIG_KPROBE_EVENT.\n");
else
pr_warning("Failed to open kprobe_events file: %s\n",
strerror(errno));
if (ret < 0)
print_warn_msg(buf, is_kprobe);
}
return ret;
}
/* Get raw string list of current kprobe_events */
static int open_kprobe_events(bool readwrite)
{
return open_probe_events("tracing/kprobe_events", readwrite, true);
}
static int open_uprobe_events(bool readwrite)
{
return open_probe_events("tracing/uprobe_events", readwrite, false);
}
/* Get raw string list of current kprobe_events or uprobe_events */
static struct strlist *get_probe_trace_command_rawlist(int fd)
{
int ret, idx;
@ -1569,36 +1641,26 @@ static int show_perf_probe_event(struct perf_probe_event *pev)
return ret;
}
/* List up current perf-probe events */
int show_perf_probe_events(void)
static int __show_perf_probe_events(int fd, bool is_kprobe)
{
int fd, ret;
int ret = 0;
struct probe_trace_event tev;
struct perf_probe_event pev;
struct strlist *rawlist;
struct str_node *ent;
setup_pager();
ret = init_vmlinux();
if (ret < 0)
return ret;
memset(&tev, 0, sizeof(tev));
memset(&pev, 0, sizeof(pev));
fd = open_kprobe_events(false);
if (fd < 0)
return fd;
rawlist = get_probe_trace_command_rawlist(fd);
close(fd);
if (!rawlist)
return -ENOENT;
strlist__for_each(ent, rawlist) {
ret = parse_probe_trace_command(ent->s, &tev);
if (ret >= 0) {
ret = convert_to_perf_probe_event(&tev, &pev);
ret = convert_to_perf_probe_event(&tev, &pev,
is_kprobe);
if (ret >= 0)
ret = show_perf_probe_event(&pev);
}
@ -1612,6 +1674,33 @@ int show_perf_probe_events(void)
return ret;
}
/* List up current perf-probe events */
int show_perf_probe_events(void)
{
int fd, ret;
setup_pager();
fd = open_kprobe_events(false);
if (fd < 0)
return fd;
ret = init_vmlinux();
if (ret < 0)
return ret;
ret = __show_perf_probe_events(fd, true);
close(fd);
fd = open_uprobe_events(false);
if (fd >= 0) {
ret = __show_perf_probe_events(fd, false);
close(fd);
}
return ret;
}
/* Get current perf-probe event names */
static struct strlist *get_probe_trace_event_names(int fd, bool include_group)
{
@ -1717,7 +1806,11 @@ static int __add_probe_trace_events(struct perf_probe_event *pev,
const char *event, *group;
struct strlist *namelist;
fd = open_kprobe_events(true);
if (pev->uprobes)
fd = open_uprobe_events(true);
else
fd = open_kprobe_events(true);
if (fd < 0)
return fd;
/* Get current event names */
@ -1829,6 +1922,8 @@ static int convert_to_probe_trace_events(struct perf_probe_event *pev,
tev->point.offset = pev->point.offset;
tev->point.retprobe = pev->point.retprobe;
tev->nargs = pev->nargs;
tev->uprobes = pev->uprobes;
if (tev->nargs) {
tev->args = zalloc(sizeof(struct probe_trace_arg)
* tev->nargs);
@ -1859,6 +1954,9 @@ static int convert_to_probe_trace_events(struct perf_probe_event *pev,
}
}
if (pev->uprobes)
return 1;
/* Currently just checking function name from symbol map */
sym = __find_kernel_function_by_name(tev->point.symbol, NULL);
if (!sym) {
@ -1894,12 +1992,18 @@ int add_perf_probe_events(struct perf_probe_event *pevs, int npevs,
int i, j, ret;
struct __event_package *pkgs;
ret = 0;
pkgs = zalloc(sizeof(struct __event_package) * npevs);
if (pkgs == NULL)
return -ENOMEM;
/* Init vmlinux path */
ret = init_vmlinux();
if (!pevs->uprobes)
/* Init vmlinux path */
ret = init_vmlinux();
else
ret = init_user_exec();
if (ret < 0) {
free(pkgs);
return ret;
@ -1971,23 +2075,15 @@ error:
return ret;
}
static int del_trace_probe_event(int fd, const char *group,
const char *event, struct strlist *namelist)
static int del_trace_probe_event(int fd, const char *buf,
struct strlist *namelist)
{
char buf[128];
struct str_node *ent, *n;
int found = 0, ret = 0;
ret = e_snprintf(buf, 128, "%s:%s", group, event);
if (ret < 0) {
pr_err("Failed to copy event.\n");
return ret;
}
int ret = -1;
if (strpbrk(buf, "*?")) { /* Glob-exp */
strlist__for_each_safe(ent, n, namelist)
if (strglobmatch(ent->s, buf)) {
found++;
ret = __del_trace_probe_event(fd, ent);
if (ret < 0)
break;
@ -1996,40 +2092,43 @@ static int del_trace_probe_event(int fd, const char *group,
} else {
ent = strlist__find(namelist, buf);
if (ent) {
found++;
ret = __del_trace_probe_event(fd, ent);
if (ret >= 0)
strlist__remove(namelist, ent);
}
}
if (found == 0 && ret >= 0)
pr_info("Info: Event \"%s\" does not exist.\n", buf);
return ret;
}
int del_perf_probe_events(struct strlist *dellist)
{
int fd, ret = 0;
int ret = -1, ufd = -1, kfd = -1;
char buf[128];
const char *group, *event;
char *p, *str;
struct str_node *ent;
struct strlist *namelist;
fd = open_kprobe_events(true);
if (fd < 0)
return fd;
struct strlist *namelist = NULL, *unamelist = NULL;
/* Get current event names */
namelist = get_probe_trace_event_names(fd, true);
if (namelist == NULL)
return -EINVAL;
kfd = open_kprobe_events(true);
if (kfd < 0)
return kfd;
namelist = get_probe_trace_event_names(kfd, true);
ufd = open_uprobe_events(true);
if (ufd >= 0)
unamelist = get_probe_trace_event_names(ufd, true);
if (namelist == NULL && unamelist == NULL)
goto error;
strlist__for_each(ent, dellist) {
str = strdup(ent->s);
if (str == NULL) {
ret = -ENOMEM;
break;
goto error;
}
pr_debug("Parsing: %s\n", str);
p = strchr(str, ':');
@ -2041,17 +2140,46 @@ int del_perf_probe_events(struct strlist *dellist)
group = "*";
event = str;
}
ret = e_snprintf(buf, 128, "%s:%s", group, event);
if (ret < 0) {
pr_err("Failed to copy event.");
free(str);
goto error;
}
pr_debug("Group: %s, Event: %s\n", group, event);
ret = del_trace_probe_event(fd, group, event, namelist);
if (namelist)
ret = del_trace_probe_event(kfd, buf, namelist);
if (unamelist && ret != 0)
ret = del_trace_probe_event(ufd, buf, unamelist);
if (ret != 0)
pr_info("Info: Event \"%s\" does not exist.\n", buf);
free(str);
if (ret < 0)
break;
}
strlist__delete(namelist);
close(fd);
error:
if (kfd >= 0) {
if (namelist)
strlist__delete(namelist);
close(kfd);
}
if (ufd >= 0) {
if (unamelist)
strlist__delete(unamelist);
close(ufd);
}
return ret;
}
/* TODO: don't use a global variable for filter ... */
static struct strfilter *available_func_filter;
@ -2068,23 +2196,8 @@ static int filter_available_functions(struct map *map __unused,
return 1;
}
int show_available_funcs(const char *target, struct strfilter *_filter)
static int __show_available_funcs(struct map *map)
{
struct map *map;
int ret;
setup_pager();
ret = init_vmlinux();
if (ret < 0)
return ret;
map = kernel_get_module_map(target);
if (!map) {
pr_err("Failed to find %s map.\n", (target) ? : "kernel");
return -EINVAL;
}
available_func_filter = _filter;
if (map__load(map, filter_available_functions)) {
pr_err("Failed to load map.\n");
return -EINVAL;
@ -2095,3 +2208,140 @@ int show_available_funcs(const char *target, struct strfilter *_filter)
dso__fprintf_symbols_by_name(map->dso, map->type, stdout);
return 0;
}
static int available_kernel_funcs(const char *module)
{
struct map *map;
int ret;
ret = init_vmlinux();
if (ret < 0)
return ret;
map = kernel_get_module_map(module);
if (!map) {
pr_err("Failed to find %s map.\n", (module) ? : "kernel");
return -EINVAL;
}
return __show_available_funcs(map);
}
static int available_user_funcs(const char *target)
{
struct map *map;
int ret;
ret = init_user_exec();
if (ret < 0)
return ret;
map = dso__new_map(target);
ret = __show_available_funcs(map);
dso__delete(map->dso);
map__delete(map);
return ret;
}
int show_available_funcs(const char *target, struct strfilter *_filter,
bool user)
{
setup_pager();
available_func_filter = _filter;
if (!user)
return available_kernel_funcs(target);
return available_user_funcs(target);
}
/*
* uprobe_events only accepts address:
* Convert function and any offset to address
*/
static int convert_name_to_addr(struct perf_probe_event *pev, const char *exec)
{
struct perf_probe_point *pp = &pev->point;
struct symbol *sym;
struct map *map = NULL;
char *function = NULL, *name = NULL;
int ret = -EINVAL;
unsigned long long vaddr = 0;
if (!pp->function) {
pr_warning("No function specified for uprobes");
goto out;
}
function = strdup(pp->function);
if (!function) {
pr_warning("Failed to allocate memory by strdup.\n");
ret = -ENOMEM;
goto out;
}
name = realpath(exec, NULL);
if (!name) {
pr_warning("Cannot find realpath for %s.\n", exec);
goto out;
}
map = dso__new_map(name);
if (!map) {
pr_warning("Cannot find appropriate DSO for %s.\n", exec);
goto out;
}
available_func_filter = strfilter__new(function, NULL);
if (map__load(map, filter_available_functions)) {
pr_err("Failed to load map.\n");
goto out;
}
sym = map__find_symbol_by_name(map, function, NULL);
if (!sym) {
pr_warning("Cannot find %s in DSO %s\n", function, exec);
goto out;
}
if (map->start > sym->start)
vaddr = map->start;
vaddr += sym->start + pp->offset + map->pgoff;
pp->offset = 0;
if (!pev->event) {
pev->event = function;
function = NULL;
}
if (!pev->group) {
char *ptr1, *ptr2;
pev->group = zalloc(sizeof(char *) * 64);
ptr1 = strdup(basename(exec));
if (ptr1) {
ptr2 = strpbrk(ptr1, "-._");
if (ptr2)
*ptr2 = '\0';
e_snprintf(pev->group, 64, "%s_%s", PERFPROBE_GROUP,
ptr1);
free(ptr1);
}
}
free(pp->function);
pp->function = zalloc(sizeof(char *) * MAX_PROBE_ARGS);
if (!pp->function) {
ret = -ENOMEM;
pr_warning("Failed to allocate memory by zalloc.\n");
goto out;
}
e_snprintf(pp->function, MAX_PROBE_ARGS, "0x%llx", vaddr);
ret = 0;
out:
if (map) {
dso__delete(map->dso);
map__delete(map);
}
if (function)
free(function);
if (name)
free(name);
return ret;
}

12
tools/perf/util/probe-event.h
View File

@ -7,7 +7,7 @@
extern bool probe_event_dry_run;
/* kprobe-tracer tracing point */
/* kprobe-tracer and uprobe-tracer tracing point */
struct probe_trace_point {
char *symbol; /* Base symbol */
char *module; /* Module name */
@ -21,7 +21,7 @@ struct probe_trace_arg_ref {
long offset; /* Offset value */
};
/* kprobe-tracer tracing argument */
/* kprobe-tracer and uprobe-tracer tracing argument */
struct probe_trace_arg {
char *name; /* Argument name */
char *value; /* Base value */
@ -29,12 +29,13 @@ struct probe_trace_arg {
struct probe_trace_arg_ref *ref; /* Referencing offset */
};
/* kprobe-tracer tracing event (point + arg) */
/* kprobe-tracer and uprobe-tracer tracing event (point + arg) */
struct probe_trace_event {
char *event; /* Event name */
char *group; /* Group name */
struct probe_trace_point point; /* Trace point */
int nargs; /* Number of args */
bool uprobes; /* uprobes only */
struct probe_trace_arg *args; /* Arguments */
};
@ -70,6 +71,7 @@ struct perf_probe_event {
char *group; /* Group name */
struct perf_probe_point point; /* Probe point */
int nargs; /* Number of arguments */
bool uprobes;
struct perf_probe_arg *args; /* Arguments */
};
@ -129,8 +131,8 @@ extern int show_line_range(struct line_range *lr, const char *module);
extern int show_available_vars(struct perf_probe_event *pevs, int npevs,
int max_probe_points, const char *module,
struct strfilter *filter, bool externs);
extern int show_available_funcs(const char *module, struct strfilter *filter);
extern int show_available_funcs(const char *module, struct strfilter *filter,
bool user);
/* Maximum index number of event-name postfix */
#define MAX_EVENT_INDEX 1024

8
tools/perf/util/symbol.c
View File

@ -2783,3 +2783,11 @@ int machine__load_vmlinux_path(struct machine *machine, enum map_type type,
return ret;
}
struct map *dso__new_map(const char *name)
{
struct dso *dso = dso__new(name);
struct map *map = map__new2(0, dso, MAP__FUNCTION);
return map;
}

1
tools/perf/util/symbol.h
View File

@ -242,6 +242,7 @@ void dso__set_long_name(struct dso *dso, char *name);
void dso__set_build_id(struct dso *dso, void *build_id);
void dso__read_running_kernel_build_id(struct dso *dso,
struct machine *machine);
struct map *dso__new_map(const char *name);
struct symbol *dso__find_symbol(struct dso *dso, enum map_type type,
u64 addr);
struct symbol *dso__find_symbol_by_name(struct dso *dso, enum map_type type,