상황
라이브디버깅 환경 (gdb) 에서 timer list 디버깅을 하려면 lx-timerlist 명령이 필요하다.
해당 명령은 source vmlinux-gdb.py 후 사용이 가능하고, vmlinux-gdb.py 는 커널소스에서 빌드해야 한다.
- https://www.kernel.org/doc/Documentation/dev-tools/gdb-kernel-debugging.rst
- https://sourceware.org/gdb/current/onlinedocs/gdb.html/Extending-
GDB.html#Extending-GDB
커스텀 빌드한 커널이 아닌, 배포판 커널을 사용해서 디버깅 해야 하는 상황이다.
목적
vmlinux-gdb.py 사용할수 있는 방법을 확인하고, timer-list 디버깅 환경을 구성한다.
사전조건
https://velog.io/@wonguk_lee/ubuntu22-커널디버깅-환경구성
환경
debugger / debuggee : Ubuntu22.04.3 LTS (jammy) 6.2.0-26-generic
- debuggee 는 부팅옵션에 nokaslr 추가필요
커널소스 확인 및 다운로드
root@user-virtual-machine:~# cat /proc/version_signature
Ubuntu 6.2.0-26.26~22.04.1-generic 6.2.13
root@user-virtual-machine:~#
root@user-virtual-machine:~# git clone https://git.launchpad.net/~ubuntu-kernel/ubuntu/+source/linux/+git/jammy -b Ubuntu-hwe-6.2-6.2.0-26.26_22.04.1 --single-branchvmlinu-gdb.py 빌드
root@user-virtual-machine:~# cd jammy
root@user-virtual-machine:~/jammy# cp /boot/config-6.2.0-26-generic .config
root@user-virtual-machine:~/jammy# make scripts_gdb디버그 심볼 이동 후 gdb 연결
root@user-virtual-machine:~/jammy# cp /usr/lib/debug/boot/vmlinux-6.2.0-26-generic vmlinux
root@user-virtual-machine:~/jammy# gdb vmlinuxsource 경로 매치
(gdb) target remote 192.168.171.1:55555
(gdb) ^C
(gdb) set substitute-path /build/linux-hwe-6.2-VRrYE0/linux-hwe-6.2-6.2.0 /root/jammy
(gdb) list start_kernel
936 asmlinkage __visible void __init __no_sanitize_address start_kernel(void)
937 {
938 char *command_line;
939 char *after_dashes;
940
941 set_task_stack_end_magic(&init_task);
942 smp_setup_processor_id();
943 debug_objects_early_init();
944 init_vmlinux_build_id();
945
(gdb)lx-symbol.py 동작확인
(gdb) source vmlinux-gdb.py
(gdb) lx-symbols
loading vmlinux
scanning for modules in /root/jammy
no module object found for 'xt_conntrack'
...
(gdb) lx-ps
TASK PID COMM
0xffffffff8321b600 0 swapper/0
0xffff8881002b9980 1 systemd
0xffff8881002be600 2 kthreadd
0xffff8881002bb300 3 rcu_gp
0xffff8881002b8000 4 rcu_par_gp
0xffff8881002bcc80 5 slub_flushwq
0xffff888100331980 6 netns
0xffff888100336600 7 kworker/0:0
0xffff888100333300 8 kworker/0:0H
0xffff888100330000 9 kworker/u256:0
0xffff888100334c80 10 mm_percpu_wq
0xffff88810033b300 11 rcu_tasks_kthre
0xffff888100338000 12 rcu_tasks_rude_
0xffff88810033cc80 13 rcu_tasks_trace
0xffff888100339980 14 ksoftirqd/0
...lx-timerlist 패치적용
vmlinux-gdb.py 로 올린 lx-timerlist 명령은 최신 커널에선 실패하는 문제가 있음
(gdb) lx-timerlist
Python Exception <class 'gdb.error'>: There is no member named next.
Error occurred in Python: There is no member named next.
(gdb) 관련해서 3개 패치가 적용됨
[PATCH 1/3] scripts/gdb: fix lx-timerlist for struct timequeue_head change
[PATCH 2/3] scripts/gdb: fix lx-timerlist for Python3
[PATCH 3/3] scripts/gdb: fix lx-timerlist for HRTIMER_MAX_CLOCK_BASES printing
적용된 Pull Request
https://github.com/torvalds/linux/commit/33afd4b76393627477e878b3b195d606e585d816#diff-54b56df924f6c983c14ed2e11a68af2b278a284de5338ad02722705d915377d6
따라서 scripts/gdb/linux/timerlist.py 를 아래와 같이 수정
root@user-virtual-machine:~/jammy# git diff
diff --git a/scripts/gdb/linux/timerlist.py b/scripts/gdb/linux/timerlist.py
index 51def847f1ef..7369e7a15405 100644
--- a/scripts/gdb/linux/timerlist.py
+++ b/scripts/gdb/linux/timerlist.py
@@ -43,8 +43,9 @@ def print_timer(rb_node, idx):
def print_active_timers(base):
- curr = base['active']['next']['node']
- curr = curr.address.cast(rbtree.rb_node_type.get_type().pointer())
+ #curr = base['active']['next']['node']
+ #curr = curr.address.cast(rbtree.rb_node_type.get_type().pointer())
+ curr = base['active']['rb_root']['rb_leftmost']
idx = 0
while curr:
yield print_timer(curr, idx)
@@ -175,7 +176,8 @@ def pr_cpumask(mask):
if 0 < extra <= 4:
chunks[0] = chunks[0][0] # Cut off the first 0
- return "".join(chunks)
+ #return "".join(chunks)
+ return "".join(str(chunks))
class LxTimerList(gdb.Command):
@@ -189,7 +191,9 @@ class LxTimerList(gdb.Command):
max_clock_bases = gdb.parse_and_eval("HRTIMER_MAX_CLOCK_BASES")
text = "Timer List Version: gdb scripts\n"
- text += "HRTIMER_MAX_CLOCK_BASES: {}\n".format(max_clock_bases)
+ #text += "HRTIMER_MAX_CLOCK_BASES: {}\n".format(max_clock_bases)
+ text += "HRTIMER_MAX_CLOCK_BASES: {}\n".format(
+ max_clock_bases.type.fields()[max_clock_bases].enumval)
text += "now at {} nsecs\n".format(ktime_get())
for cpu in cpus.each_online_cpu():
root@user-virtual-machine:~/jammy# 이후 lx-timerlist 명령을 수행하면 정상적으로 timer list 정보확인이 되는걸 확인
(gdb) lx-timerlist
Timer List Version: gdb scripts
HRTIMER_MAX_CLOCK_BASES: 8
now at 66353363090 nsecs
cpu: 0
clock 0:
.base: 0xffff888139e23340
.index: 0
.resolution: 1 nsecs
.get_time: 0xffffffff811d98b0 <ktime_get>
.offset: 0 nsecs
active timers:
#0: <0xffff888139e23860>, tick_sched_timer, S:01
# expires at 68256671194-68256671194 nsecs [in 1903308104 to 1903308104 nsecs]
#1: <0xffff888139e23ac0>, watchdog_timer_fn, S:01
# expires at 69352671194-69352671194 nsecs [in 2999308104 to 2999308104 nsecs]
#2: <0xffffc90005a9f978>, hrtimer_wakeup, S:01
# expires at 3630940828468-3631040828468 nsecs [in 3564587465378 to 3564687465378 nsecs]
.expires_next : 68256671194 nsecs
.hres_active : 1
.nr_events : 3391
.nr_retries : 0
.nr_hangs : 0
.max_hang_time : 0
.nohz_mode : NOHZ_MODE_HIGHRES
.last_tick : 67024671194 nsecs
.tick_stopped : 1
.idle_jiffies : 4294908723
.idle_calls : 1579
.idle_sleeps : 1579
.idle_entrytime : 67090866644 nsecs
.idle_waketime : 67090858755 nsecs
.idle_exittime : 67021270498 nsecs
.idle_sleeptime : 58615795174 nsecs
.iowait_sleeptime: 588835871 nsecs
.last_jiffies : 4294908740
.next_timer : 68256671194
.idle_expires : 68256671194 nsecs
jiffies: 4294908744
clock 1:
.base: 0xffff888139e23380
...
관심분야 : Filesystem, Data structure, user/kernel IPC
2개의 댓글
2024년 5월 20일
https://blog.csdn.net/u013836909/article/details/136439290
source ./vmlinux-gdb.py 로 사용 필요
답글 달기
make scripts_gdb 는 내부적으로 git 을 부르고 있으므로, 인터넷 연결 필요