initcall_level이란

디바이스 드라이버나 커널 드라이버가 초기화되는 레벨
빌트인 타입으로 등록된 모든 디바이스 드라이버가 초기화됨

• 빌트인 타입 : 작성한 디바이스 드라이버가 커널 이미지에 포함이 되어서 커널의 기능이 하나로 부팅이 됨
• 모듈 타입 : insmod 명령을 사용해서 디바이스 드라이버를 모듈타입으로 설치

실전 프로젝트에서 initcall_level

• init콜을 어느 레벨에서 설정할 지가 중요하다.
  • late_init에서 arch_init으로 드라이버를 초기화하는 시점 변경하는 경우가 생길수도
• 부팅 관련 이슈를 분석하기 위한 기반 지식
  - 작성된 드라이버가 어느 initcall_level 에서 초기화되는지 점검 필요

예시

• __init : 일정 섹션에 이 함수가 위치하게되고 해당 섹션에 존재하는 함수들은 리눅스 커널이 부팅하는 과정에서 이 함수를 읽어서 초기화하는 루틴을 실행

• late_initcall() : 이닛 콜을 실행을 하겠다.

  • late레벨에서 실행하겠다.

init call level

initcall level signature

• initcall_level: level=console
  initcall_level: level=early
  initcall_level: level=pure
  initcall_level: level=core
  initcall_level: level=postcore
  initcall_level: level=arch
  initcall_level: level=subsys
  initcall_level: level=fs
  initcall_level: level=device
  initcall_level: level=late

• 입문자 입장에서 디바이스 드라이버 : 페리페럴 디바이스를 제어하고 컨트롤 하는 인터페이스
  • 리눅스 커널 내부의 여러 서브시스템(ftrace , FS, cpu arch에 의존적인 디바이스 드라이버)는 console ~ device level에서 실행된다
  • 대신 페리페럴 디바이스 드라이버의 대부분의 이닛콜은 late레벨에서 실행된다.

콘솔 레벨

• 콘솔 이닛에서 trace_initcall_level이 호출
  • start , finish 들의 이벤트가 실행될 때 ftrace이벤트를 트레이싱해준다.

init call level names

init/main.c
static const char *initcall_level_names[] __initdata = {
"pure",
"core",
"postcore",
"arch",
"subsys",
"fs",
"device",
"late",
};

init call 선언부

arch init call

• 다른 매크로들도 같은 패턴 => 레벨에 맞게 initcall이 수행됨

init call names

• 아래의 레벨들이 어느 코드에서 실행이 되나?
• [level] 에 확인되는 문자열을 수행한다.

initcall_level: level=console

• console init 확인
• nop 트레이싱을 사용해서 콘솔 레벨에 해당하는 init call 메시지

initcall_level: level=early

• trace_init_flags_sys_enter 부터 시작

systemd-1 [000] ..... 0.004843: initcall_level: level=early
systemd-1 [000] ..... 0.004849: initcall_start: func=trace_init_flags_sys_enter+0x0/0x2c
systemd-1 [000] ..... 0.004851: initcall_finish: func=trace_init_flags_sys_enter+0x0/0x2c ret=0
systemd-1 [000] ..... 0.004855: initcall_start: func=trace_init_flags_sys_exit+0x0/0x2c
systemd-1 [000] ..... 0.004856: initcall_finish: func=trace_init_flags_sys_exit+0x0/0x2c ret=0

initcall_level: level=pure

systemd-1 [000] ..... 0.034750: initcall_level: level=pure
systemd-1 [000] ..... 0.034759: <stack trace>
=> __traceiter_initcall_level+0x3c/0x5c <ffffffe1920143a0>
=> trace_initcall_level+0x60/0x98 <ffffffe192ba2288>
=> kernel_init_freeable+0x23c/0x2cc <ffffffe192fe12c8>
=> kernel_init+0x2c/0x140 <ffffffe192bb2f1c>
=> ret_from_fork+0x10/0x20 <ffffffe1920166f0>
systemd-1 [000] ..... 0.034763: initcall_start: func=bpf_jit_charge_init+0x0/0x54
systemd-1 [000] ..... 0.034765: <stack trace>
=> do_one_initcall+0x1b0/0x2a0 <ffffffe192014ee0>
=> kernel_init_freeable+0x25c/0x2cc <ffffffe192fe12e8>
=> kernel_init+0x2c/0x140 <ffffffe192bb2f1c>
=> ret_from_fork+0x10/0x20 <ffffffe1920166f0>
systemd-1[000] .....0.034768: initcall_finish: func=bpf_jit_charge_init+0x0/0x54 ret=0

systemd-1 [000] ..... 0.034549: initcall_level: level=pure
systemd-1 [000] ..... 0.034558: initcall_start: func=bpf_jit_charge_init+0x0/0x54 // 이닛콜 수행
systemd-1 [000] ..... 0.034561: initcall_finish: func=bpf_jit_charge_init+0x0/0x54 ret=0
systemd-1 [000] ..... 0.034566: initcall_start: func=ipc_ns_init+0x0/0x4c
systemd-1 [000] ..... 0.034575: initcall_finish: func=ipc_ns_init+0x0/0x4c ret=0
systemd-1 [000] ..... 0.034579: initcall_start: func=init_mmap_min_addr+0x0/0x28
systemd-1 [000] ..... 0.034580: initcall_finish: func=init_mmap_min_addr+0x0/0x28 ret=0

initcall_level: level=core

systemd-1 [000] ..... 0.035334: initcall_level: level=core
systemd-1 [000] ..... 0.035336: <stack trace>
=> __traceiter_initcall_level+0x3c/0x5c <ffffffe1920143a0>
=> trace_initcall_level+0x60/0x98 <ffffffe192ba2288>
=> kernel_init_freeable+0x23c/0x2cc <ffffffe192fe12c8>
=> kernel_init+0x2c/0x140 <ffffffe192bb2f1c>
=> ret_from_fork+0x10/0x20 <ffffffe1920166f0>
systemd-1 [000] ..... 0.035338: initcall_start: func=fpsimd_init+0x0/0xf8
systemd-1 [000] ..... 0.035340: <stack trace>
=> do_one_initcall+0x1b0/0x2a0 <ffffffe192014ee0>
=> kernel_init_freeable+0x25c/0x2cc <ffffffe192fe12e8>
=> kernel_init+0x2c/0x140 <ffffffe192bb2f1c>
=> ret_from_fork+0x10/0x20 <ffffffe1920166f0>
systemd-1 [000] ..... 0.035346: initcall_finish: func=fpsimd_init+0x0/0xf8 ret=0

systemd-1 [000] 0.035083: initcall_level: level=core
systemd-1 [000] 0.035085: initcall_start: func=fpsimd_init+0x0/0xf8
systemd-1 [000] 0.035090: initcall_finish: func=fpsimd_init+0x0/0xf8 ret=0
systemd-1 [000] 0.035094: initcall_start: func=tagged_addr_init+0x0/0x40
systemd-1 [000] 0.035110: initcall_finish: func=tagged_addr_init+0x0/0x40 ret=0
systemd-1 [000] 0.035113: initcall_start: func=enable_mrs_emulation+0x0/0x30
systemd-1 [000] 0.035117: initcall_finish: func=enable_mrs_emulation+0x0/0x30 ret=0
systemd-1 [000] 0.035121: initcall_start: func=init_amu_fie+0x0/0x58
systemd-1 [000] 0.035124: initcall_finish: func=init_amu_fie+0x0/0x58 ret=0
systemd-1 [000] 0.035127: initcall_start: func=arm64_efi_rt_init+0x0/0xb8
systemd-1 [000] 0.035129: initcall_finish: func=arm64_efi_rt_init+0x0/0xb8 ret=0
systemd-1 [000] 0.035132: initcall_start: func=armv8_deprecated_init+0x0/0x164
systemd-1 [000] 0.035217: initcall_finish: func=armv8_deprecated_init+0x0/0x164 ret=0

• arm 아키텍처에 의존적인 동작을 수행한다.

• init call level을 알게되면 리눅스 커널의 디바이스 드라이버 부팅 흐름을 잡을 수 있다.

initcall_level: level=postcore

systemd-1 [000] ..... 0.050948: initcall_level: level=postcore
systemd-1 [000] ..... 0.050950: <stack trace>
=> __traceiter_initcall_level+0x3c/0x5c <ffffffe1920143a0>
=> trace_initcall_level+0x60/0x98 <ffffffe192ba2288>
=> kernel_init_freeable+0x23c/0x2cc <ffffffe192fe12c8>
=> kernel_init+0x2c/0x140 <ffffffe192bb2f1c>
=> ret_from_fork+0x10/0x20 <ffffffe1920166f0>
systemd-1 [000] ..... 0.050951: initcall_start: func=debug_monitors_init+0x0/0x44
systemd-1 [000] ..... 0.050952: <stack trace>
=> do_one_initcall+0x1b0/0x2a0 <ffffffe192014ee0>
=> kernel_init_freeable+0x25c/0x2cc <ffffffe192fe12e8>
=> kernel_init+0x2c/0x140 <ffffffe192bb2f1c>
=> ret_from_fork+0x10/0x20 <ffffffe1920166f0>
systemd-1 [000] ..... 0.051185: initcall_finish: func=debug_monitors_init+0x0/0x44 ret=0

systemd-1 [000] ..... 0.050745: initcall_level: level=postcore
systemd-1 [000] ..... 0.050747: initcall_start: func=debug_monitors_init+0x0/0x44
systemd-1 [000] ..... 0.050979: initcall_finish: func=debug_monitors_init+0x0/0x44 ret=0
systemd-1 [000] ..... 0.050983: initcall_start: func=irq_sysfs_init+0x0/0x110
systemd-1 [000] ..... 0.051277: initcall_finish: func=irq_sysfs_init+0x0/0x110 ret=0
systemd-1 [000] ..... 0.051281: initcall_start: func=dma_atomic_pool_init+0x0/0x160
systemd-1 [000] ..... 0.054890: initcall_finish: func=dma_atomic_pool_init+0x0/0x160 ret=0
systemd-1 [000] ..... 0.054895: initcall_start: func=audit_init+0x0/0x150
systemd-1 [000] ..... 0.055243: initcall_finish: func=audit_init+0x0/0x150 ret=0

• 다음과 같은 init call을 확인할 수 있다.

initcall_level: level=arch

systemd-1 [000] ..... 0.056602: initcall_level: level=arch
systemd-1 [000] ..... 0.056604: <stack trace>
=> __traceiter_initcall_level+0x3c/0x5c <ffffffe1920143a0>
=> trace_initcall_level+0x60/0x98 <ffffffe192ba2288>
=> kernel_init_freeable+0x23c/0x2cc <ffffffe192fe12c8>
=> kernel_init+0x2c/0x140 <ffffffe192bb2f1c>
=> ret_from_fork+0x10/0x20 <ffffffe1920166f0>
systemd-1 [000] ..... 0.056606: initcall_start:
func=reserve_memblock_reserved_regions+0x0/0x164
systemd-1 [000] ..... 0.056607: <stack trace>
=> do_one_initcall+0x1b0/0x2a0 <ffffffe192014ee0>
=> kernel_init_freeable+0x25c/0x2cc <ffffffe192fe12e8>
=> kernel_init+0x2c/0x140 <ffffffe192bb2f1c>
=> ret_from_fork+0x10/0x20 <ffffffe1920166f0>
systemd-1 [000] ..... 0.056680: initcall_finish:
func=reserve_memblock_reserved_regions+0x0/0x164 ret=0

systemd-1 [000] ..... 0.056274: initcall_level: level=arch
systemd-1 [000] ..... 0.056277: initcall_start:
func=reserve_memblock_reserved_regions+0x0/0x164
systemd-1 [000] ..... 0.056349: initcall_finish:
func=reserve_memblock_reserved_regions+0x0/0x164 ret=0
systemd-1 [000] ..... 0.056353: initcall_start: func=aarch32_alloc_vdso_pages+0x0/0xf4
systemd-1 [000] ..... 0.056367: initcall_finish: func=aarch32_alloc_vdso_pages+0x0/0xf4 ret=0
systemd-1 [000] ..... 0.056370: initcall_start: func=vdso_init+0x0/0xf8
systemd-1 [000] ..... 0.056372: initcall_finish: func=vdso_init+0x0/0xf8 ret=0
systemd-1 [000] ..... 0.056376: initcall_start: func=arch_hw_breakpoint_init+0x0/0x100
systemd-1 [000] ..... 0.056562: initcall_finish: func=arch_hw_breakpoint_init+0x0/0x100 ret=0
systemd-1 [000] ..... 0.056567: initcall_start: func=adjust_protection_map+0x0/0x50
systemd-1 [000] ..... 0.056568: initcall_finish: func=adjust_protection_map+0x0/0x50 ret=0
systemd-1 [000] ..... 0.056571: initcall_start: func=asids_update_limit+0x0/0xbc
systemd-1 [000] ..... 0.056593: initcall_finish: func=asids_update_limit+0x0/0xbc

• arm에 의존적인 init콜이 나온다.

initcall_level: level=subsys

systemd-1 [000] ..... 0.111425: initcall_level: level=subsys
systemd-1 [000] ..... 0.111427: <stack trace>
=> __traceiter_initcall_level+0x3c/0x5c <ffffffe1920143a0>
=> trace_initcall_level+0x60/0x98 <ffffffe192ba2288>
=> kernel_init_freeable+0x23c/0x2cc <ffffffe192fe12c8>
=> kernel_init+0x2c/0x140 <ffffffe192bb2f1c>
=> ret_from_fork+0x10/0x20 <ffffffe1920166f0>
systemd-1 [000] ..... 0.111430: initcall_start: func=topology_init+0x0/0x88
systemd-1 [000] ..... 0.111432: <stack trace>
=> do_one_initcall+0x1b0/0x2a0 <ffffffe192014ee0>
=> kernel_init_freeable+0x25c/0x2cc <ffffffe192fe12e8>
=> kernel_init+0x2c/0x140 <ffffffe192bb2f1c>
=> ret_from_fork+0x10/0x20 <ffffffe1920166f0>
systemd-1 [000] ..... 0.111969: initcall_finish: func=topology_init+0x0/0x88 ret=0

systemd-1 [000] ..... 0.111384: initcall_level: level=subsys
systemd-1 [000] ..... 0.111386: initcall_start: func=topology_init+0x0/0x88
systemd-1 [000] ..... 0.111927: initcall_finish: func=topology_init+0x0/0x88 ret=0
systemd-1 [000] ..... 0.111932: initcall_start: func=kaslr_init+0x0/0xe8
systemd-1 [000] ..... 0.111955: initcall_finish: func=kaslr_init+0x0/0xe8 ret=0
systemd-1 [000] ..... 0.111958: initcall_start: func=register_mte_tcf_preferred_sysctl+0x0/0xc4
systemd-1 [000] ..... 0.111959: initcall_finish: func=register_mte_tcf_preferred_sysctl+0x0/0xc4
ret=0
systemd-1 [000] ..... 0.111963: initcall_start: func=uid_cache_init+0x0/0xd4
systemd-1 [000] ..... 0.111999: initcall_finish: func=uid_cache_init+0x0/0xd4 ret=0
systemd-1 [000] ..... 0.112003: initcall_start: func=param_sysfs_init+0x0/0x1e8
systemd-1 [000] ..... 0.138463: initcall_finish: func=param_sysfs_init+0x0/0x1e8 ret=0
systemd-1 [000] ..... 0.138468: initcall_start: func=user_namespace_sysctl_init+0x0/0xfc
systemd-1 [000] ..... 0.138509: initcall_finish: func=user_namespace_sysctl_init+0x0/0xfc ret=0
systemd-1 [000] ..... 0.138522: initcall_start: func=proc_schedstat_init+0x0/0x4c
systemd-1 [000] ..... 0.138536: initcall_finish: func=proc_schedstat_init+0x0/0x4c ret=0
systemd-1 [000] ..... 0.138540: initcall_start: func=pm_sysrq_init+0x0/0x34
systemd-1 [000] ..... 0.138611: initcall_finish: func=pm_sysrq_init+0x0/0x34 ret=0

initcall_level: level=fs

systemd-1 [001] ..... 0.156380: initcall_level: level=fs
systemd-1 [001] ..... 0.156383: <stack trace>
=> __traceiter_initcall_level+0x3c/0x5c <ffffffe1920143a0>
=> trace_initcall_level+0x60/0x98 <ffffffe192ba2288>
=> kernel_init_freeable+0x23c/0x2cc <ffffffe192fe12c8>
=> kernel_init+0x2c/0x140 <ffffffe192bb2f1c>
=> ret_from_fork+0x10/0x20 <ffffffe1920166f0>
systemd-1 [001] ..... 0.156385: initcall_start: func=create_debug_debugfs_entry+0x0/0x40
systemd-1 [001] ..... 0.156386: <stack trace>
=> do_one_initcall+0x1b0/0x2a0 <ffffffe192014ee0>
=> kernel_init_freeable+0x25c/0x2cc <ffffffe192fe12e8>
=> kernel_init+0x2c/0x140 <ffffffe192bb2f1c>
=> ret_from_fork+0x10/0x20 <ffffffe1920166f0>
systemd-1 [001] ..... 0.156405: initcall_finish: func=create_debug_debugfs_entry+0x0/0x40 ret=0

• vfs와 관련된 api가 실행된다. func=create_debug_debugfs_entry+0x0/0x40 ret=0

systemd-1 [001] ..... 0.151843: initcall_level: level=fs
systemd-1 [001] ..... 0.151845: initcall_start: func=create_debug_debugfs_entry+0x0/0x40
systemd-1 [001] ..... 0.151862: initcall_finish: func=create_debug_debugfs_entry+0x0/0x40 ret=0
systemd-1 [001] ..... 0.151866: initcall_start: func=iomem_init_inode+0x0/0xbc
systemd-1 [001] ..... 0.151925: initcall_finish: func=iomem_init_inode+0x0/0xbc ret=0
systemd-1 [001] ..... 0.151929: initcall_start: func=clocksource_done_booting+0x0/0x6c
systemd-1 [001] ..... 0.152025: initcall_finish: func=clocksource_done_booting+0x0/0x6c ret=0
systemd-1 [001] ..... 0.152033: initcall_start: func=tracer_init_tracefs+0x0/0xe8
systemd-1 [001] ..... 0.152057: initcall_finish: func=tracer_init_tracefs+0x0/0xe8 ret=0
systemd-1 [001] ..... 0.152060: initcall_start: func=init_trace_printk_function_export+0x0/0x44
systemd-1 [001] ..... 0.152263: initcall_finish: func=init_trace_printk_function_export+0x0/0x44
ret=0
systemd-1 [001] ..... 0.152268: initcall_start: func=init_graph_tracefs+0x0/0x44
systemd-1 [001] ..... 0.152280: initcall_finish: func=init_graph_tracefs+0x0/0x44 ret=0

initcall_level: level=device

systemd-1 [001] ..... 0.174953: initcall_level: level=device
systemd-1 [001] ..... 0.174955: <stack trace>
=> __traceiter_initcall_level+0x3c/0x5c <ffffffe1920143a0>
=> trace_initcall_level+0x60/0x98 <ffffffe192ba2288>
=> kernel_init_freeable+0x23c/0x2cc <ffffffe192fe12c8>
=> kernel_init+0x2c/0x140 <ffffffe192bb2f1c>
=> ret_from_fork+0x10/0x20 <ffffffe1920166f0>
systemd-1 [001] ..... 0.174957: initcall_start: func=register_arm64_panic_block+0x0/0x38
systemd-1 [001] ..... 0.174959: <stack trace>
=> do_one_initcall+0x1b0/0x2a0 <ffffffe192014ee0>
=> kernel_init_freeable+0x25c/0x2cc <ffffffe192fe12e8>
=> kernel_init+0x2c/0x140 <ffffffe192bb2f1c>
=> ret_from_fork+0x10/0x20 <ffffffe1920166f0>
systemd-1 [001] ..... 0.174964: initcall_finish: func=register_arm64_panic_block+0x0/0x38 ret=0

systemd-1 [001] ..... 0.170535: initcall_level: level=device
systemd-1 [001] ..... 0.170538: initcall_start: func=register_arm64_panic_block+0x0/0x38
systemd-1 [001] ..... 0.170543: initcall_finish: func=register_arm64_panic_block+0x0/0x38 ret=0
systemd-1 [001] ..... 0.170546: initcall_start: func=cpuinfo_regs_init+0x0/0xdc
systemd-1 [001] ..... 0.170951: initcall_finish: func=cpuinfo_regs_init+0x0/0xdc ret=0
systemd-1 [001] ..... 0.170955: initcall_start: func=aarch32_el0_sysfs_init+0x0/0x48
systemd-1 [001] ..... 0.170957: initcall_finish: func=aarch32_el0_sysfs_init+0x0/0x48 ret=0
systemd-1 [001] ..... 0.170960: initcall_start: func=armv8_pmu_driver_init+0x0/0x30
systemd-1 [001] ..... 0.173199: initcall_finish: func=armv8_pmu_driver_init+0x0/0x30 ret=0
systemd-1 [001] ..... 0.173206: initcall_start: func=arm_init+0x0/0x34
systemd-1 [001] ..... 0.176233: initcall_finish: func=arm_init+0x0/0x34 ret=0

initcall_level: level=late

systemd-1 [001] ..... 1.595505: initcall_level: level=late
systemd-1 [001] ..... 1.595507: <stack trace>
=> __traceiter_initcall_level+0x3c/0x5c <ffffffe1920143a0>
=> trace_initcall_level+0x60/0x98 <ffffffe192ba2288>
=> kernel_init_freeable+0x23c/0x2cc <ffffffe192fe12c8>
=> kernel_init+0x2c/0x140 <ffffffe192bb2f1c>
=> ret_from_fork+0x10/0x20 <ffffffe1920166f0>
systemd-1 [001] ..... 1.595508: initcall_start: func=kernel_do_mounts_initrd_sysctls_init+0x0/0x40
systemd-1 [001] ..... 1.595510: <stack trace>
=> do_one_initcall+0x1b0/0x2a0 <ffffffe192014ee0>
=> kernel_init_freeable+0x25c/0x2cc <ffffffe192fe12e8>
=> kernel_init+0x2c/0x140 <ffffffe192bb2f1c>
=> ret_from_fork+0x10/0x20 <ffffffe1920166f0>
systemd-1 [001] ..... 1.595522: initcall_finish: func=kernel_do_mounts_initrd_sysctls_init+0x0/0x40
ret=0

systemd-1 [000] ..... 1.576323: initcall_level: level=late
systemd-1 [000] ..... 1.576325: initcall_start: func=kernel_do_mounts_initrd_sysctls_init+0x0/0x40
systemd-1 [000] ..... 1.576339: initcall_finish:
func=kernel_do_mounts_initrd_sysctls_init+0x0/0x40 ret=0
systemd-1 [000] ..... 1.576342: initcall_start: func=kernel_panic_sysctls_init+0x0/0x40
systemd-1 [000] ..... 1.576355: initcall_finish: func=kernel_panic_sysctls_init+0x0/0x40 ret=0
systemd-1 [000] ..... 1.576360: initcall_start: func=kernel_panic_sysfs_init+0x0/0x40
systemd-1 [000] ..... 1.576370: initcall_finish: func=kernel_panic_sysfs_init+0x0/0x40 ret=0
systemd-1 [000] ..... 1.576373: initcall_start: func=kernel_exit_sysctls_init+0x0/0x40
systemd-1 [000] ..... 1.576378: initcall_finish: func=kernel_exit_sysctls_init+0x0/0x40 ret=0
systemd-1 [000] ..... 1.576382: initcall_start: func=kernel_exit_sysfs_init+0x0/0x40
systemd-1 [000] ..... 1.576387: initcall_finish: func=kernel_exit_sysfs_init+0x0/0x40 ret=0
systemd-1 [000] ..... 1.576391: initcall_start: func=reboot_ksysfs_init+0x0/0x94
systemd-1 [000] ..... 1.576412: initcall_finish: func=reboot_ksysfs_init+0x0/0x94 ret=0

• 모듈 타입으로 로딩이되는 디바이스 드라이버가 init 콜 수행한 메시지