Understanding the Linux Kernel: The Linux Kernel Startup
Understanding the Linux Kernel: The Linux Kernel Startup
By Jesús Espino · May 11, 2026 · 36 min read · Tags: #Linux #Kernel #Operating Systems URL: https://internals-for-interns.com/posts/linux-kernel-startup/
First in a series on Linux kernel internals. Focused on x86_64. Goal is a "useful map" rather than an exhaustive tour.
Have you ever wondered what really happens between the moment you press the power button and the moment your login screen shows up? That gap — usually some seconds — hides one of the most intricate initialization sequences in computing.
Imagine We're Setting Up a Space Colony
Picture a barren planet. We send a small advance team in a dropship. Their mission: turn this rock into a working colony, and do it before life support runs out.
The advance team can't just unload everyone and start hosting town hall meetings. They have to do things in a very specific order. First the basics, then map the terrain, then bring up the construction equipment, then proper governance, then wake the colonists. The bootloader is the dropship. Your computer is the barren planet. The advance team is the execution of the startup code. By the end of this article, that advance team will literally have transformed itself into the standby maintenance crew while a brand-new civilian government takes over.
High-level flow: Bootloader → Assembly Entry (startup_64) → Early C (x86_64_start_kernel) → Arch Setup (setup_arch) → Core Subsystems (start_kernel) → Threading (rest_init) → Finalization (kernel_init) → User Space (/sbin/init).
The Handoff: What the Bootloader Hands Us
The CPU is already running in one of several modes. UEFI puts us straight in Long Mode; legacy BIOS usually leaves us in Protected Mode. Memory is awkward: the kernel was loaded low in RAM (typically around 0x1000000) but compiled to run at a high virtual address (something like 0xffffffff81000000). Firmware also gives us an E820 map, ACPI tables, and a bag of boot parameters. No console, no allocator, no interrupts, no log.
Phase 1: The Assembly Trampoline
Unpacking the Kernel First
The bzImage is compressed. A tiny decompressor in arch/x86/boot/compressed/ unpacks the real kernel image. It also picks a random base address — KASLR (Kernel Address Space Layout Randomization).
Into the Real Kernel
Legacy 32-bit boot lands at startup_32 and has to climb to 64-bit Long Mode (build identity-mapped page table, flip the bit, turn paging on, jump to startup_64). Modern UEFI jumps straight to startup_64. Either way, every path converges there. The first thing: point the stack pointer at a small pre-allocated buffer and install a minimal GDT and IDT.
A Detour for Encrypted Hardware
Some AMD CPUs can transparently encrypt RAM — SME (Secure Memory Encryption), with SEV for VMs. Intel's analogue is TDX; the all-RAM-one-key feature is TME. The kernel turns encryption on right now — you can't go back and encrypt data already written in the clear.
Did the Lander Even Survive?
verify_cpu (arch/x86/kernel/verify_cpu.S) checks Long Mode support, SSE2, other CPU features. If something essential is missing, we stop here. This is why you can't run a 64-bit kernel on a 32-bit CPU.
The Address Mismatch Problem
The kernel was loaded at one address but compiled for another. Page table fixup: startup_64 calls a C helper, __startup_64() (via position-independent thunk __pi___startup_64), which computes the difference between "where the code thinks it is" and "where we actually landed" and patches the page table entries by that offset.
Jumping into C
The kernel switches the CPU over to the patched page tables and jumps to its first real C function, x86_64_start_kernel. We get C only barely: no allocator, no console, no library calls. Just C with raw pointers and discipline.
Phase 2: Early C Initialization
x86_64_start_kernel lives in arch/x86/kernel/head64.c. cr4_init_shadow() caches CR4, reset_early_page_tables() discards the identity mapping.
Wiping Down the Workbenches
clear_bss() zeroes the .bss section. In a normal C program the runtime would do this — but we are the runtime.
KASAN, the Safety Inspector with No Office Yet
sme_early_init() finishes wiring up memory encryption. Then KASAN (Kernel Address Sanitizer) needs a huge region of shadow memory but we don't have an allocator. The trick: point the entire shadow region at a single zero page so instrumented code reads zeros instead of crashing. Real shadow comes later.
Emergency Procedures Before You Need Them
If something goes wrong now, the CPU needs an IDT entry to know who to call. Three failures in a row = triple fault = silent reboot. idt_setup_early_handler() installs minimal handlers for Page Faults, General Protection Faults, etc.
Saving the Bootloader's Notes
copy_bootdata() copies command line, memory map, initrd location into kernel-owned boot_params. sanitize_boot_params() zeroes fields the bootloader had no business filling in.
Patching the CPU Itself
load_ucode_bsp() patches the CPU's own microcode. Some CPU bugs (Spectre, MDS) are fixed by microcode updates. The advance team finishes updating the lander's firmware before going anywhere.
Phase 3: Hardware Discovery and Memory Setup
setup_arch() in arch/x86/kernel/setup.c. The orbital survey turned into ground truth.
Cataloging the Team's Skills
early_cpu_init() calls CPUID and dumps answers into boot_cpu_data. The kernel's lookup for "do we have feature X?" Used later for self-patching.
Reading the Survey
e820__memory_setup() pulls in the firmware's memory map and sanitizes it. e820__memblock_setup() feeds the clean map into memblock, a primitive early allocator that tracks "this range is free, this range is reserved." parse_setup_data() reads any extra bootloader notes (RNG seed, firmware fixups). early_ioremap_init() lets the kernel peek at firmware tables (ACPI) before a real allocator exists.
The First Radio: Early Printk
parse_early_param() walks boot arguments and dispatches the ones marked early — if you passed earlyprintk=serial,ttyS0, the handler now sets up a minimal serial driver. The kernel can dribble out messages over a low-power short-range radio.
Where Are We Running?
efi_init() hooks up UEFI runtime services. dmi_setup() reads firmware tables describing motherboard, vendor, BIOS — useful for per-vendor quirks. init_hypervisor_platform() checks for KVM, Xen, Hyper-V, VMware. Early ACPI pass. Low-memory trampoline reserved for waking other cores in Phase 6.
Mapping All of RAM
kernel_randomize_memory() picks randomized base addresses for the direct map. reserve_brk() sets aside scratch. init_mem_mapping() builds the kernel direct map: a giant region of virtual addresses where every physical page gets a fixed virtual address.
Roping Off the Important Areas
reserve_initrd() and arch_reserve_crashkernel() reserve those regions in memblock. With a real memory map, kasan_init() sets up KASAN's real shadow memory, replacing the zero-page placeholder.
Phase 4: Core Subsystems Come Online
start_kernel() in init/main.c is the longest function in the boot path. Four things it's trying to do:
- Make this CPU usable.
- Make memory and tracing usable.
- Make time and concurrency usable.
- Make processes, files, networks, and security usable.
Make This CPU Usable
set_task_stack_end_magic() arms a stack tripwire. CPU is identified, debugging frameworks armed, build ID extracted, basic cgroup state set up so the boot thread has a group. Then interrupts are turned off — almost the entire init sequence runs with interrupts disabled until local_irq_enable() much later. boot_cpu_init() officially marks this CPU as online. The Linux version banner gets pushed into the printk ring buffer.
Architecture Setup, Static Keys, Command Line
setup_arch() actually runs here. jump_label_init() and static_call_init(): the kernel patches its own code so if (some_flag) checks become NOP/JMP at boot, and many indirect calls become direct. Runtime feature flags cost zero CPU cycles in hot paths.
early_security_init() brings up LSM (Linux Security Module) just enough to install the most fundamental security hooks. setup_command_line() keeps two copies — one verbatim for /proc/cmdline, one the parser is allowed to chew up. setup_per_cpu_areas() allocates per-CPU memory regions — every CPU gets its own private locker. parse_args() applies command-line options to whatever registered for them.
Memory Comes Alive: mm_core_init()
This is the big one. Until now memblock has been our only allocator. After mm_core_init(), kmalloc(), vmalloc(), and alloc_pages() all work.
Three layered allocators come up:
- Buddy allocator — page-grained, groups physical pages into power-of-two blocks.
memblock_free_all()walks every pagememblocknever reserved and gives it to the buddy. - Slab allocator (SLUB default) — sits on top, specializes in objects (
task_struct,inode). Same trick as Go'smspan. - vmalloc allocator — big region contiguous in virtual address space when physical can't be.
Around this: kmemleak_init (memory-leak detector), poking_init (machinery to safely rewrite kernel code), ftrace_init, early_trace_init (tracing).
The Scheduler: sched_init()
Going in, exactly one thread — init_task, riding since the first assembly instruction. By the time sched_init() returns, every CPU has a runqueue. Scheduler classes are ordered stop > deadline > rt > fair > idle. Wrong order = panic. init_task is quietly turned into the boot CPU's idle task. The advance team has just been issued their long-term assignment — they don't know it yet. Everything after this in start_kernel() is the idle task pretending to be init code.
(Caveat: scheduler can switch tasks on the boot CPU here, but can't load-balance across CPUs yet — that's sched_init_smp() in Phase 6.)
RCU, Workqueues, and Tracing
workqueue_init_early() brings up the workqueue subsystem — but doesn't hire any workers yet. Scheduled work piles up like maintenance tickets at a help desk that hasn't hired anyone. Don't confuse with runqueue: runqueues hold tasks; workqueues hold pending function calls.
rcu_init() brings up RCU (Read-Copy-Update). On data read constantly but updated rarely (network device list, routing table), readers proceed lock-free; writers build a new copy and swap a pointer to publish.
trace_init() finishes wiring up the kernel's tracing infrastructure.
Interrupts and Time
early_irq_init, init_IRQ: per-interrupt bookkeeping, talk to local APIC / GIC. Hardware could deliver interrupts but they're still globally masked.
tick_init() arms the periodic heartbeat (with optional tickless mode for idle CPUs). timers_init(), hrtimers_init(): ordinary and high-resolution timers. softirq_init() sets up tasklets.
timekeeping_init() reads the real-time clock and anchors wall-clock and monotonic clocks. time_init() lets the architecture register a higher-quality clock source (TSC on x86).
random_init() brings up the RNG, mixing early entropy (RDRAND, bootloader seed, timing of boot events).
Setting Up the Stack Canary
kfence_init() arms KFENCE (low-overhead memory-safety detector complementing KASAN). boot_init_stack_canary() installs a real random value as the stack canary; until now it held a predictable placeholder.
Turning Interrupts On
perf_event_init, legacy profiling, call_function_init (cross-CPU IPI mechanism). Then local_irq_enable(). The colony just turned on its alarm system and loudspeakers.
The Real Console
console_init() walks each registered console driver. Every message piled up in the printk ring buffer since the very first pr_notice gets drained to the console here — which is why your boot log appears in a sudden burst rather than a steady trickle.
CPU Finalization and Self-Patching
ACPI subsystem online. calibrate_delay() produces the BogoMIPS number. Then arch_cpu_finalize_init(): full CPUID-driven feature discovery, chooses the right Spectre/Meltdown/MDS mitigations, sets up the FPU, calls alternative_instructions(). The kernel patches every ALTERNATIVE macro site, swapping in instructions that match this CPU's actual feature set. The kernel has been runtime-rewritten to be optimal for this exact CPU. Got AVX-512? Some memcpy paths get rewritten. The same vmlinux file boots optimally on dozens of CPU generations.
Slab Caches for Processes, Files, Namespaces, Networking
Dozens of small calls, each creating the slab caches for a specific subsystem:
fork_init()(withcred_init,signals_init,pagecache_init) —task_structcache. HR is open.vfs_caches_init()— VFS, the virtual filesystem layer. Mount tiny in-memory rootfs.proc_root_init()brings up/proc.cgroup_init(),security_init()(full LSM),net_ns_init()and siblings (pidfs_init,nsfs_init,cpuset_init,mem_cgroup_init) — namespace plumbing that lets containers exist.
Phase 5: From Single-Threaded to Multitasking
rest_init() in init/main.c. Never returns.
Spawning the First Real Processes
Two kernel threads:
- PID 1 (
kernel_init) — destined toexec()into userspace and become init (/sbin/init,systemd, OpenRC). - PID 2 (
kthreadd) — kernel thread daemon. Everykthread_create()from now until shutdown gets routed here.
Subtle ordering trick: kernel_init is spawned first to get PID 1, then kthreadd. kernel_init immediately blocks on wait_for_completion(&kthreadd_done) so it can't actually start working until kthreadd is alive. PIDs come out right, nobody starts working until the dispatcher is open.
(PID 1 is initially pinned to the boot CPU until sched_init_smp() in Phase 6.)
The Idle Loop, and the Quiet Disappearance of the Boot Thread
rest_init() calls schedule_preempt_disabled() — yields the CPU. The scheduler picks one of the new threads (most likely kernel_init) and context-switches. When control eventually finds its way back, cpu_startup_entry(CPUHP_ONLINE) runs do_idle() forever.
That switch is the magic moment. The original thread of execution — running since startup_64 in raw assembly, through setup_arch, start_kernel, every subsystem init — has just permanently become the boot CPU's idle task (PID 0). It only runs again when literally nothing else wants the CPU.
Phase 6: Finishing Up and Launching User Space
kernel_init (PID 1, still in kernel mode), most work in kernel_init_freeable().
Hiring the Workers and Waking Up the Other CPUs
workqueue_init() — kworker/... threads come online and chew through the backlog. smp_init() sends inter-processor interrupts (IPIs) to wake parked cores; each goes through its own simplified boot before joining the idle loop. sched_init_smp() builds the scheduling domains the load balancer uses. Real multitasking begins.
Loading the Drivers and Filesystems (initcalls)
page_alloc_init_late() finishes deferred page setup. do_initcalls() runs thousands of initcalls grouped into ordered levels (early, core, subsys, fs, device, late). This is where the boot log fills up: PCI: Using ACPI for IRQ routing, ata1: SATA max UDMA/133 ..., ext4 filesystem driver registered. wait_for_initramfs(), console_on_rootfs() (opens /dev/console as stdin/stdout/stderr).
Mounting the Root Filesystem
prepare_namespace() mounts whatever you specified with root=. On modern distros: initramfs unpacked into a tmpfs, /init script loads drivers, mounts the real root, and pivots to it.
Throwing Out the Setup Crew's Toolboxes
All __init-tagged functions are now dead weight. free_initmem() reclaims the section, typically a few megabytes: Freeing unused kernel memory: 2048K.
Locking the Doors Behind Us
mark_readonly() flips rodata to actually read-only at the page-table level. pti_finalize() finalizes PTI (Page Table Isolation) — Meltdown mitigation. You can't lock the doors while you're still installing them — now that they're installed, throw the deadbolts.
Executing Init: The Final Handoff
The kernel walks candidate paths:
rdinit=from command line (typically/initinside initramfs)init=from command line — if set explicitly and program fails, kernel panicsCONFIG_DEFAULT_INITif any/sbin/init, then/etc/init, then/bin/init/bin/shlast-ditch fallback
run_init_process() calls kernel_execve(). Replaces the kernel thread with the userspace program. Same PID, same task, but now running unprivileged code. If every path fails: No working init found.
Summary
Six carefully ordered phases:
- Assembly — Decompress (KASLR), Long Mode, verify CPU, page-table fixup, jump into C.
- Early C — Clear BSS, KASAN placeholder, IDT, save boot params, microcode patch.
setup_arch— Detect CPU, parse firmware memory map, identify machine, build direct map, finalize KASAN.start_kernel—mm_core_init, scheduler, RCU, time, IRQs (then enable), real console, self-patch for this CPU, slab caches.rest_init— Spawn PID 1 (kernel_init) and PID 2 (kthreadd). Original boot thread becomes PID 0 idle.kernel_init— Wake other CPUs, initcalls, mount root, free__init, lock down,exec()userspace init.
Next article in the series: how userspace actually talks to the kernel — system calls.