# Understanding the Linux Kernel: The Linux Kernel Startup # Understanding the Linux Kernel: The Linux Kernel Startup **By Jesús Espino** · May 11, 2026 · 36 min read · Tags: #Linux #Kernel #Operating Systems URL: https://internals-for-interns.com/posts/linux-kernel-startup/ > First in a series on Linux kernel internals. Focused on **x86_64**. Goal is a "useful map" rather than an exhaustive tour. Have you ever wondered what really happens between the moment you press the power button and the moment your login screen shows up? That gap — usually some seconds — hides one of the most intricate initialization sequences in computing. ## Imagine We're Setting Up a Space Colony Picture a barren planet. We send a small **advance team** in a dropship. Their mission: turn this rock into a working colony, and do it before life support runs out. The advance team can't just unload everyone and start hosting town hall meetings. They have to do things in a very specific order. First the basics, then map the terrain, then bring up the construction equipment, then proper governance, then wake the colonists. The bootloader is the dropship. Your computer is the barren planet. The advance team is the execution of the startup code. By the end of this article, that advance team will literally have transformed itself into the standby maintenance crew while a brand-new civilian government takes over. High-level flow: Bootloader → Assembly Entry (`startup_64`) → Early C (`x86_64_start_kernel`) → Arch Setup (`setup_arch`) → Core Subsystems (`start_kernel`) → Threading (`rest_init`) → Finalization (`kernel_init`) → User Space (`/sbin/init`). ## The Handoff: What the Bootloader Hands Us The CPU is already running in one of several modes. UEFI puts us straight in Long Mode; legacy BIOS usually leaves us in Protected Mode. Memory is awkward: the kernel was loaded low in RAM (typically around `0x1000000`) but compiled to run at a high virtual address (something like `0xffffffff81000000`). Firmware also gives us an **E820 map**, ACPI tables, and a bag of boot parameters. No console, no allocator, no interrupts, no log. ## Phase 1: The Assembly Trampoline ### Unpacking the Kernel First The `bzImage` is compressed. A tiny **decompressor** in `arch/x86/boot/compressed/` unpacks the real kernel image. It also picks a random base address — **KASLR** (Kernel Address Space Layout Randomization). ### Into the Real Kernel Legacy 32-bit boot lands at `startup_32` and has to climb to 64-bit Long Mode (build identity-mapped page table, flip the bit, turn paging on, jump to `startup_64`). Modern UEFI jumps straight to `startup_64`. Either way, every path converges there. The first thing: point the stack pointer at a small pre-allocated buffer and install a minimal **GDT** and **IDT**. ### A Detour for Encrypted Hardware Some AMD CPUs can transparently encrypt RAM — **SME** (Secure Memory Encryption), with **SEV** for VMs. Intel's analogue is **TDX**; the all-RAM-one-key feature is **TME**. The kernel turns encryption on right now — you can't go back and encrypt data already written in the clear. ### Did the Lander Even Survive? `verify_cpu` (`arch/x86/kernel/verify_cpu.S`) checks Long Mode support, SSE2, other CPU features. If something essential is missing, we stop here. This is why you can't run a 64-bit kernel on a 32-bit CPU. ### The Address Mismatch Problem The kernel was loaded at one address but compiled for another. **Page table fixup**: `startup_64` calls a C helper, `__startup_64()` (via position-independent thunk `__pi___startup_64`), which computes the difference between "where the code thinks it is" and "where we actually landed" and patches the page table entries by that offset. ### Jumping into C The kernel switches the CPU over to the patched page tables and jumps to its first real C function, `x86_64_start_kernel`. We get C only barely: no allocator, no console, no library calls. Just C with raw pointers and discipline. ## Phase 2: Early C Initialization `x86_64_start_kernel` lives in `arch/x86/kernel/head64.c`. `cr4_init_shadow()` caches CR4, `reset_early_page_tables()` discards the identity mapping. ### Wiping Down the Workbenches `clear_bss()` zeroes the `.bss` section. In a normal C program the runtime would do this — but we *are* the runtime. ### KASAN, the Safety Inspector with No Office Yet `sme_early_init()` finishes wiring up memory encryption. Then **KASAN** (Kernel Address Sanitizer) needs a huge region of shadow memory but we don't have an allocator. The trick: point the entire shadow region at a single zero page so instrumented code reads zeros instead of crashing. Real shadow comes later. ### Emergency Procedures Before You Need Them If something goes wrong now, the CPU needs an **IDT** entry to know who to call. Three failures in a row = **triple fault** = silent reboot. `idt_setup_early_handler()` installs minimal handlers for Page Faults, General Protection Faults, etc. ### Saving the Bootloader's Notes `copy_bootdata()` copies command line, memory map, initrd location into kernel-owned `boot_params`. `sanitize_boot_params()` zeroes fields the bootloader had no business filling in. ### Patching the CPU Itself `load_ucode_bsp()` patches the **CPU's own microcode**. Some CPU bugs (Spectre, MDS) are fixed by microcode updates. The advance team finishes updating the lander's firmware before going anywhere. ## Phase 3: Hardware Discovery and Memory Setup `setup_arch()` in `arch/x86/kernel/setup.c`. The orbital survey turned into ground truth. ### Cataloging the Team's Skills `early_cpu_init()` calls **CPUID** and dumps answers into `boot_cpu_data`. The kernel's lookup for "do we have feature X?" Used later for self-patching. ### Reading the Survey `e820__memory_setup()` pulls in the firmware's memory map and sanitizes it. `e820__memblock_setup()` feeds the clean map into **`memblock`**, a primitive early allocator that tracks "this range is free, this range is reserved." `parse_setup_data()` reads any extra bootloader notes (RNG seed, firmware fixups). `early_ioremap_init()` lets the kernel peek at firmware tables (ACPI) before a real allocator exists. ### The First Radio: Early Printk `parse_early_param()` walks boot arguments and dispatches the ones marked `early` — if you passed `earlyprintk=serial,ttyS0`, the handler now sets up a minimal serial driver. The kernel can dribble out messages over a low-power short-range radio. ### Where Are We Running? `efi_init()` hooks up UEFI runtime services. `dmi_setup()` reads firmware tables describing motherboard, vendor, BIOS — useful for per-vendor quirks. `init_hypervisor_platform()` checks for KVM, Xen, Hyper-V, VMware. Early ACPI pass. Low-memory trampoline reserved for waking other cores in Phase 6. ### Mapping All of RAM `kernel_randomize_memory()` picks randomized base addresses for the direct map. `reserve_brk()` sets aside scratch. `init_mem_mapping()` builds the **kernel direct map**: a giant region of virtual addresses where every physical page gets a fixed virtual address. ### Roping Off the Important Areas `reserve_initrd()` and `arch_reserve_crashkernel()` reserve those regions in `memblock`. With a real memory map, `kasan_init()` sets up KASAN's real shadow memory, replacing the zero-page placeholder. ## Phase 4: Core Subsystems Come Online `start_kernel()` in `init/main.c` is the longest function in the boot path. Four things it's trying to do: 1. Make this CPU usable. 2. Make memory and tracing usable. 3. Make time and concurrency usable. 4. Make processes, files, networks, and security usable. ### Make This CPU Usable `set_task_stack_end_magic()` arms a stack tripwire. CPU is identified, debugging frameworks armed, build ID extracted, basic cgroup state set up so the boot thread has a group. Then interrupts are turned off — almost the entire init sequence runs with interrupts disabled until `local_irq_enable()` much later. `boot_cpu_init()` officially marks this CPU as online. The Linux version banner gets pushed into the printk ring buffer. ### Architecture Setup, Static Keys, Command Line `setup_arch()` actually runs here. `jump_label_init()` and `static_call_init()`: the kernel patches its own code so `if (some_flag)` checks become NOP/JMP at boot, and many indirect calls become direct. Runtime feature flags cost zero CPU cycles in hot paths. `early_security_init()` brings up **LSM (Linux Security Module)** just enough to install the most fundamental security hooks. `setup_command_line()` keeps two copies — one verbatim for `/proc/cmdline`, one the parser is allowed to chew up. `setup_per_cpu_areas()` allocates per-CPU memory regions — every CPU gets its own private locker. `parse_args()` applies command-line options to whatever registered for them. ### Memory Comes Alive: `mm_core_init()` This is the big one. Until now `memblock` has been our only allocator. After `mm_core_init()`, `kmalloc()`, `vmalloc()`, and `alloc_pages()` all work. Three layered allocators come up: - **Buddy allocator** — page-grained, groups physical pages into power-of-two blocks. `memblock_free_all()` walks every page `memblock` never reserved and gives it to the buddy. - **Slab allocator** (SLUB default) — sits on top, specializes in objects (`task_struct`, `inode`). Same trick as Go's `mspan`. - **vmalloc allocator** — big region contiguous in virtual address space when physical can't be. Around this: `kmemleak_init` (memory-leak detector), `poking_init` (machinery to safely rewrite kernel code), `ftrace_init`, `early_trace_init` (tracing). ### The Scheduler: `sched_init()` Going in, exactly one thread — `init_task`, riding since the first assembly instruction. By the time `sched_init()` returns, every CPU has a runqueue. Scheduler classes are ordered `stop > deadline > rt > fair > idle`. Wrong order = panic. **`init_task` is quietly turned into the boot CPU's idle task.** The advance team has just been issued their long-term assignment — they don't know it yet. Everything after this in `start_kernel()` is the idle task pretending to be init code. (Caveat: scheduler can switch tasks on the boot CPU here, but can't load-balance across CPUs yet — that's `sched_init_smp()` in Phase 6.) ### RCU, Workqueues, and Tracing `workqueue_init_early()` brings up the workqueue subsystem — but doesn't hire any workers yet. Scheduled work piles up like maintenance tickets at a help desk that hasn't hired anyone. **Don't confuse with runqueue**: runqueues hold tasks; workqueues hold pending function calls. `rcu_init()` brings up **RCU (Read-Copy-Update)**. On data read constantly but updated rarely (network device list, routing table), readers proceed lock-free; writers build a new copy and swap a pointer to publish. `trace_init()` finishes wiring up the kernel's tracing infrastructure. ### Interrupts and Time `early_irq_init`, `init_IRQ`: per-interrupt bookkeeping, talk to local APIC / GIC. Hardware *could* deliver interrupts but they're still globally masked. `tick_init()` arms the periodic heartbeat (with optional tickless mode for idle CPUs). `timers_init()`, `hrtimers_init()`: ordinary and high-resolution timers. `softirq_init()` sets up tasklets. `timekeeping_init()` reads the **real-time clock** and anchors wall-clock and monotonic clocks. `time_init()` lets the architecture register a higher-quality clock source (TSC on x86). `random_init()` brings up the RNG, mixing early entropy (RDRAND, bootloader seed, timing of boot events). ### Setting Up the Stack Canary `kfence_init()` arms KFENCE (low-overhead memory-safety detector complementing KASAN). `boot_init_stack_canary()` installs a real random value as the stack canary; until now it held a predictable placeholder. ### Turning Interrupts On `perf_event_init`, legacy profiling, `call_function_init` (cross-CPU IPI mechanism). Then `local_irq_enable()`. The colony just turned on its alarm system and loudspeakers. ### The Real Console `console_init()` walks each registered console driver. Every message piled up in the printk ring buffer since the very first `pr_notice` gets drained to the console here — which is why your boot log appears in a sudden burst rather than a steady trickle. ### CPU Finalization and Self-Patching ACPI subsystem online. `calibrate_delay()` produces the **BogoMIPS** number. Then `arch_cpu_finalize_init()`: full CPUID-driven feature discovery, chooses the right Spectre/Meltdown/MDS mitigations, sets up the FPU, calls **`alternative_instructions()`**. The kernel patches every `ALTERNATIVE` macro site, swapping in instructions that match this CPU's actual feature set. **The kernel has been runtime-rewritten to be optimal for this exact CPU.** Got AVX-512? Some `memcpy` paths get rewritten. The same `vmlinux` file boots optimally on dozens of CPU generations. ### Slab Caches for Processes, Files, Namespaces, Networking Dozens of small calls, each creating the slab caches for a specific subsystem: - `fork_init()` (with `cred_init`, `signals_init`, `pagecache_init`) — `task_struct` cache. HR is open. - `vfs_caches_init()` — VFS, the virtual filesystem layer. Mount tiny in-memory rootfs. `proc_root_init()` brings up `/proc`. - `cgroup_init()`, `security_init()` (full LSM), `net_ns_init()` and siblings (`pidfs_init`, `nsfs_init`, `cpuset_init`, `mem_cgroup_init`) — namespace plumbing that lets containers exist. ## Phase 5: From Single-Threaded to Multitasking `rest_init()` in `init/main.c`. Never returns. ### Spawning the First Real Processes Two kernel threads: - **PID 1** (`kernel_init`) — destined to `exec()` into userspace and become **init** (`/sbin/init`, `systemd`, OpenRC). - **PID 2** (`kthreadd`) — kernel thread daemon. Every `kthread_create()` from now until shutdown gets routed here. Subtle ordering trick: `kernel_init` is spawned first to get PID 1, then `kthreadd`. `kernel_init` immediately blocks on `wait_for_completion(&kthreadd_done)` so it can't actually start working until `kthreadd` is alive. PIDs come out right, nobody starts working until the dispatcher is open. (PID 1 is initially pinned to the boot CPU until `sched_init_smp()` in Phase 6.) ### The Idle Loop, and the Quiet Disappearance of the Boot Thread `rest_init()` calls `schedule_preempt_disabled()` — yields the CPU. The scheduler picks one of the new threads (most likely `kernel_init`) and context-switches. When control eventually finds its way back, `cpu_startup_entry(CPUHP_ONLINE)` runs `do_idle()` forever. **That switch is the magic moment.** The original thread of execution — running since `startup_64` in raw assembly, through `setup_arch`, `start_kernel`, every subsystem init — has just permanently become the boot CPU's **idle task (PID 0)**. It only runs again when literally nothing else wants the CPU. ## Phase 6: Finishing Up and Launching User Space `kernel_init` (PID 1, still in kernel mode), most work in `kernel_init_freeable()`. ### Hiring the Workers and Waking Up the Other CPUs `workqueue_init()` — `kworker/...` threads come online and chew through the backlog. `smp_init()` sends **inter-processor interrupts (IPIs)** to wake parked cores; each goes through its own simplified boot before joining the idle loop. `sched_init_smp()` builds the **scheduling domains** the load balancer uses. *Real multitasking begins.* ### Loading the Drivers and Filesystems (initcalls) `page_alloc_init_late()` finishes deferred page setup. `do_initcalls()` runs thousands of **initcalls** grouped into ordered levels (`early`, `core`, `subsys`, `fs`, `device`, `late`). This is where the boot log fills up: `PCI: Using ACPI for IRQ routing`, `ata1: SATA max UDMA/133 ...`, `ext4 filesystem driver registered`. `wait_for_initramfs()`, `console_on_rootfs()` (opens `/dev/console` as stdin/stdout/stderr). ### Mounting the Root Filesystem `prepare_namespace()` mounts whatever you specified with `root=`. On modern distros: **initramfs** unpacked into a `tmpfs`, `/init` script loads drivers, mounts the real root, and pivots to it. ### Throwing Out the Setup Crew's Toolboxes All `__init`-tagged functions are now dead weight. `free_initmem()` reclaims the section, typically a few megabytes: `Freeing unused kernel memory: 2048K`. ### Locking the Doors Behind Us `mark_readonly()` flips `rodata` to actually read-only at the page-table level. `pti_finalize()` finalizes **PTI (Page Table Isolation)** — Meltdown mitigation. You can't lock the doors while you're still installing them — now that they're installed, throw the deadbolts. ### Executing Init: The Final Handoff The kernel walks candidate paths: 1. `rdinit=` from command line (typically `/init` inside initramfs) 2. `init=` from command line — **if set explicitly and program fails, kernel panics** 3. `CONFIG_DEFAULT_INIT` if any 4. `/sbin/init`, then `/etc/init`, then `/bin/init` 5. `/bin/sh` last-ditch fallback `run_init_process()` calls `kernel_execve()`. **Replaces** the kernel thread with the userspace program. Same PID, same task, but now running unprivileged code. If every path fails: `No working init found.` ## Summary Six carefully ordered phases: 1. **Assembly** — Decompress (KASLR), Long Mode, verify CPU, page-table fixup, jump into C. 2. **Early C** — Clear BSS, KASAN placeholder, IDT, save boot params, microcode patch. 3. **`setup_arch`** — Detect CPU, parse firmware memory map, identify machine, build direct map, finalize KASAN. 4. **`start_kernel`** — `mm_core_init`, scheduler, RCU, time, IRQs (then enable), real console, self-patch for this CPU, slab caches. 5. **`rest_init`** — Spawn PID 1 (`kernel_init`) and PID 2 (`kthreadd`). Original boot thread becomes PID 0 idle. 6. **`kernel_init`** — Wake other CPUs, initcalls, mount root, free `__init`, lock down, `exec()` userspace init. Next article in the series: how userspace actually talks to the kernel — system calls.