ftrace¶

1 挂载debugfs¶

大部分linux发行版已经挂载到了/sys/kernel/debug或/sys/kernel/tracing目录下，所以不需要此操作。

[root@centos7 ~]# mkdir -p /debug
[root@centos7 ~]# mount  -t  debugfs  debugfs  /debug

2 最小示例¶

进入trace目录：cd /sys/kernel/debug/tracing
指定跟踪资源：echo function >current_tracer（跟踪函数）
打开trace开关：echo 1 > tracing_on
查看trace得到的日志信息：cat trace |head -20

[root@centos7 tracing]# cd /sys/kernel/debug/tracing
[root@centos7 tracing]# echo function >current_tracer
[root@centos7 tracing]# echo 1 > tracing_on 
[root@centos7 tracing]# cat trace |head -20
# tracer: function
#
# entries-in-buffer/entries-written: 102609/17086024   #P:2
#
#                                _-----=> irqs-off/BH-disabled
#                               / _----=> need-resched
#                              | / _---=> hardirq/softirq
#                              || / _--=> preempt-depth
#                              ||| / _-=> migrate-disable
#                              |||| /     delay
#           TASK-PID     CPU#  |||||  TIMESTAMP  FUNCTION
#              | |         |   |||||     |         |
          <idle>-0       [000] d..2.   880.780380: cpuidle_enter_state <-cpuidle_enter
        vmtoolsd-798     [001] ...1.   881.852240: sockfd_lookup_light <-__sys_bind
        vmtoolsd-798     [001] ...1.   881.852241: __fdget <-sockfd_lookup_light
        vmtoolsd-798     [001] ...1.   881.852241: __fget_light <-__fdget
        vmtoolsd-798     [001] ...1.   881.852241: __rcu_read_lock <-__fget_light
        vmtoolsd-798     [001] ...1.   881.852241: __rcu_read_unlock <-__fget_light
        vmtoolsd-798     [001] ...1.   881.852241: move_addr_to_kernel.part.0 <-__sys_bind
        vmtoolsd-798     [001] ...1.   881.852242: __check_object_size <-move_addr_to_kernel.part.0

3 trace目录下关键文件说明¶

3.1 available_tracers¶

功能：显示可用的tracer
支持参数：
- function_graph：函数调用栈
- function：函数
- nop：不跟踪任务tracer
示例

[root@centos7 tracing]# cat available_tracers 
blk function_graph wakeup_dl wakeup_rt wakeup function nop

3.2 current_tracer¶

功能：设置或显示当前tracer
可用参数：available_tracers提供的参数
示例

[root@centos7 tracing]# echo function_graph > current_tracer 
[root@centos7 tracing]# cat current_tracer 
function_graph

3.3 tracing_on¶

功能：设置或显示是否开启ring buffer，以支持tracer
可用参数：
- 0：关闭
- 1：开启
示例

[root@centos7 tracing]# echo 1 >tracing_on

3.4 trace、trace_pipe¶

title: trace 和 trace_pipe 有什么不同？
- trace从ring buffer读取数据，而不会清除ring buffer里面数据
- trace_pipe类似消费者，从ring buffer读取数据后，ring buffer里面的数据也没了

功能：用于从ring buffer中读取tracer日志信息
示例

[root@centos7 tracing]# cat trace |head -20
# tracer: function
#
# entries-in-buffer/entries-written: 102609/17086024   #P:2
#
#                                _-----=> irqs-off/BH-disabled
#                               / _----=> need-resched
#                              | / _---=> hardirq/softirq
#                              || / _--=> preempt-depth
#                              ||| / _-=> migrate-disable
#                              |||| /     delay
#           TASK-PID     CPU#  |||||  TIMESTAMP  FUNCTION
#              | |         |   |||||     |         |
          <idle>-0       [000] d..2.   880.780380: cpuidle_enter_state <-cpuidle_enter
        vmtoolsd-798     [001] ...1.   881.852240: sockfd_lookup_light <-__sys_bind
        vmtoolsd-798     [001] ...1.   881.852241: __fdget <-sockfd_lookup_light
        vmtoolsd-798     [001] ...1.   881.852241: __fget_light <-__fdget
        vmtoolsd-798     [001] ...1.   881.852241: __rcu_read_lock <-__fget_light
        vmtoolsd-798     [001] ...1.   881.852241: __rcu_read_unlock <-__fget_light
        vmtoolsd-798     [001] ...1.   881.852241: move_addr_to_kernel.part.0 <-__sys_bind
        vmtoolsd-798     [001] ...1.   881.852242: __check_object_size <-move_addr_to_kernel.part.0

3.5 available_filter_functions¶

功能：显示ftrace已经处理并可以跟踪的函数。可以用在 set_ftrace_filter, set_ftrace_notrace,set_graph_function, or set_graph_notrace中
示例

[root@centos7 tracing]# cat available_filter_functions |wc -l
48793
[root@centos7 tracing]# cat available_filter_functions |grep open|head -10
fake_panic_fops_open
severities_coverage_open
mce_chrdev_open
mtrr_open
microcode_open
msr_open
cpuid_open
memtype_seq_open
clear_warn_once_fops_open
open_softirq

3.6 set_ftrace_filter¶

功能：设置函数过滤器，只跟踪此函数

3.7 set_ftrace_notrace¶

功能：设置函数过滤器，不跟踪此函数

3.8 set_graph_function¶

功能：设置function_graph过滤器，只跟踪此function_graph（set_ftrace_filter依然有效）

3.9 set_graph_notrace¶

功能：设置function_graph过滤器，不跟踪此function_graph

3.10 set_ftrace_pid¶

功能：设置函数进程过滤器，只跟踪此进程

3.11 set_ftrace_notrace_pid¶

功能：设置哈桑农户进程过滤器，不跟踪此进程

3.12 set_event_pid¶

功能：设置事件进程过滤器，只跟踪此进程

3.13 set_event_notrace_pid¶

功能：设置事件进程过滤器，不跟踪此进程

4 trace脚本¶

[root@centos7 trace]# cat trace.sh 
#!/bin/bash
tracer=
cd /sys/kernel/debug/tracing
for i in $(cat available_tracers);do
  if [ "$1" = "$i" ];then
    tracer="$1"
    break
  fi
done
if [ "$tracer" = "" ];then
  echo "param not valid"
  cat available_tracers
  exit 1
fi
echo  $tracer > current_tracer
echo vfs_open > set_ftrace_filter
echo 1 > tracing_on
temp=$(ls)
echo 0 > tracing_on
cat trace

执行示例

[root@centos7 trace]# sh trace.sh function_graph
# tracer: function_graph
#
# CPU  DURATION                  FUNCTION CALLS
# |     |   |                     |   |   |   |
   1)   2.008 us    |  vfs_open();
   1)   0.344 us    |  vfs_open();
   1)   1.143 us    |  vfs_open();
   1)   0.323 us    |  vfs_open();
   1)   0.755 us    |  vfs_open();
   1)   0.554 us    |  vfs_open();
   1)   0.557 us    |  vfs_open();
   1)   0.666 us    |  vfs_open();
   1)   0.566 us    |  vfs_open();
   1)   0.581 us    |  vfs_open();
   1)   0.555 us    |  vfs_open();
   1)   1.026 us    |  vfs_open();
   1)   1.289 us    |  vfs_open();