Skip to content
TheChatArchiveThe Chat Archive
GPT 5.1· openaiHugging Face Frontier Reasoning Traces

Implement a virtual memory manager in C for a small operating system kernel.

  1. User

    Implement a virtual memory manager in C for a small operating system kernel.

  2. Assistant

    Outlining virtual memory manager

    The user might be asking for an implementation of a virtual memory manager that's reusable. I think I should start by mentioning some basic assumptions like a 32-bit x86 architecture with paging and 4 KiB pages. If the user needs other architecture, that can be adapted.

    Then, I can show the header file with necessary data structures and APIs. I want to include a basic physical frame allocator but might need to stub it, so I could mention prototypes for pmm_alloc_frame() and pmm_free_frame() without going into too much detail.Structuring the memory management

    I’ll assume we're using a simple buddy system or bitmap for memory allocation, but I won’t dive deeply into implementing the physical memory manager (PMM). I think showing an extremely simple linear frame allocator could work, though it may seem unrealistic. It might be better to provide an interface for the physical memory manager and simply comment that users can "plug in their own."

    I want to focus on the virtual memory manager functions and include minimal dependencies while noting that we must manipulate specific architecture bits to enable paging. I'll include some inline assembly while reminding that it needs to run in ring 0. Since it's open-ended, I should provide a minimal example, keeping in mind that it's only for kernel usage with 32-bit GCC and marking hardware-specific assembly clearly.Designing a virtual memory manager API

    Many OS development novices often read resources like Bran's "kernel" and typically map the kernel to 0xC0000000, but that's not necessary. Instead, we can identity-map the first 4 MiB and place the kernel there. Since they asked for a virtual memory manager, it’s essential to provide an ability to map and unmap arbitrary pages, so I’ll design a generic API.

    I’ll start with assumptions such as 4KB pages and a 32-bit x86 architecture without PAE, using uint32_t for addresses.

    For data structures, I’ll define page table entries and directory entries, including macros for constants. We also need to ensure that page directory entries point to the physical address of the page tables while defining various flags for page attributes.Creating a page directory function

    I need to implement a function that creates a new page directory with all page directory entries initialized to zero. This requires physical frames for page table structures, which I can allocate using pmm_alloc_frame(). Since my VM uses identity mapping for the kernel region, I can consider physical addresses to be the same as virtual addresses for low memory.

    I’ll maintain that the first 4MB of physical memory is identity-mapped, where page structures reside. While advanced VMM designs involve a "kernel heap," I need to keep things general for a small OS kernel and ensure proper early-kernel bridging. However, the user did not clarify if paging is already set up, so enabling paging might need to be part of the implementation.Implementing a virtual memory manager

    I’m thinking through the process of starting in 32-bit protected mode with paging disabled. First, I'll create a page directory and tables that identity-map from 0x0 to a certain memory size. Once I load CR3 and enable paging by setting the PG bit in CR0, we can use the VMM for mapping new pages.

    I’ll define some functions for initializing memory management and allocating pages, relying on a simple physical memory manager (PMM) with a bump allocator. It's important to ensure memory constraints are respected while maintaining clarity in the code.

    Next, I’ll lay out how to identity-map the memory range and handle page tables correctly, including functions that manage page mappings and retrieval of physical addresses. I need to ensure that the implementation is practical and flexible for users adapting the code for their own needs.Mapping and enabling the memory manager

    I’m working on setting up the virtual memory manager (VMM). I’ll start by implementing the functions to switch the page directory and enable paging. First, I need to ensure that paging is off before initializing everything. During initialization, I’ll call physical memory initialization, allocate the kernel directory, and identity-map the necessary memory regions.

    It’s important to consider performance; mapping the entire memory size might lead to heavy usage, especially with large memory sizes. A plan to limit initial mapping to a sensible size, like 128MB, is a good approach. I’ll also create allocation functions for virtual memory space, focusing on clarity and keeping the code well-structured with comments, especially for novices.

    In the header file, I'll define all the necessary types and functions for the VMM and ensure proper inline assembly for operations like loading page directories and enabling paging. For error handling, the page fault function will log faults, which I’ll provide a skeleton for. I need to keep in mind the overall organization and implementation details, ensuring everything is clear for users who want to adapt the example into their projects.Implementing a Physical Memory Manager

    I'm working on the physical memory manager (PMM) implementation. I'll define next_free_frame and max_frame for tracking allocated frames. In pmm_init, I'll calculate the starting frame based on kernel_end and the maximum frame based on the provided memory size.

    Currently, I won't implement pmm_free_frame, as the focus is on demonstrating mapping and unmapping functionality. We need to ensure the frames for page tables are not overlapping with others, but I'm confident we can manage it since we're allocating frames after kernel_end. It's crucial that I call pmm_init before vmm_init, but I’ll handle that in my setup.

    Next, I'll create a title and sections for my documentation about the 32-bit x86 virtual memory manager, explaining assumptions and key parts with clear code blocks. I'll strive for clarity while keeping it concise, and I should include necessary typedefs to make it more versatile and specific for kernel implementations. Below is a minimal but complete example of a 32‑bit x86 paging‑based virtual memory manager in C suitable for a small kernel.

    It:

    • Sets up paging (4 KiB pages, 2‑level page tables, non‑PAE).
    • Identity‑maps a region of physical memory so the kernel can continue to run.
    • Provides functions to map/unmap pages and translate virtual→physical.
    • Includes a very simple physical frame allocator (bump pointer).
    • Includes helpers to switch page directories and a page fault handler stub.

    You will need to adapt it to your own environment (linker script, bootloader, etc.), but the core logic is here.


    1. Header: vmm.h

    #ifndef VMM_H
    #define VMM_H
    
    /* Basic fixed‑width types for a 32‑bit kernel. Adjust if you already
     * have your own type definitions. */
    typedef unsigned int  uint32_t;
    typedef unsigned int  uintptr_t;
    typedef unsigned long size_t;
    
    #define PAGE_SIZE 4096U
    #define PAGE_MASK 0xFFFFF000U
    
    /* Page flags (PTE & PDE) for x86 */
    enum {
        PAGE_PRESENT        = 0x001,  /* Present in memory */
        PAGE_WRITE          = 0x002,  /* Writable */
        PAGE_USER           = 0x004,  /* User accessible */
        PAGE_WRITE_THROUGH  = 0x008,
        PAGE_CACHE_DISABLE  = 0x010,
        PAGE_ACCESSED       = 0x020,
        PAGE_DIRTY          = 0x040,
        PAGE_GLOBAL         = 0x100
    };
    
    typedef uint32_t page_table_entry_t;
    typedef uint32_t page_directory_entry_t;
    
    typedef struct page_table {
        page_table_entry_t entries[1024];
    } __attribute__((aligned(PAGE_SIZE))) page_table_t;
    
    typedef struct page_directory {
        page_directory_entry_t entries[1024];
    } __attribute__((aligned(PAGE_SIZE))) page_directory_t;
    
    /* Initialize physical and virtual memory management.
     * memory_size_bytes: total usable RAM size reported by bootloader.
     */
    void vmm_init(uint32_t memory_size_bytes);
    
    /* Switch to given page directory (must be physical addressable). */
    void vmm_switch_directory(page_directory_t *dir);
    
    /* Get currently active page directory. */
    page_directory_t *vmm_get_current_directory(void);
    
    /* Map a virtual address to a physical address in a page directory. 
     * 'flags' should include at least PAGE_WRITE, optionally PAGE_USER, etc.
     * PAGE_PRESENT is implied and added by vmm_map().
     */
    void vmm_map(page_directory_t *dir,
                 uintptr_t virt_addr,
                 uintptr_t phys_addr,
                 uint32_t flags);
    
    /* Unmap a virtual address in given directory (does NOT free physical frame
     * in this simple implementation – add that if you implement free frames).
     */
    void vmm_unmap(page_directory_t *dir, uintptr_t virt_addr);
    
    /* Translate virtual address to physical address in given directory.
     * Returns 0 if not mapped.
     */
    uintptr_t vmm_virt_to_phys(page_directory_t *dir, uintptr_t virt_addr);
    
    /* Page fault handler stub – integrate into your IDT/ISR code.
     * 'faulting_address' is usually read from CR2.
     */
    void vmm_handle_page_fault(uint32_t faulting_address,
                               uint32_t error_code);
    
    #endif /* VMM_H */
    

    2. Implementation: vmm.c

    #include "vmm.h"
    
    /* ================================
     *  Simple support functions
     * ================================ */
    
    /* Minimal memset to avoid libc dependency in a freestanding kernel. */
    static void *k_memset(void *dst, int value, size_t count)
    {
        unsigned char *p = (unsigned char *)dst;
        while (count--) {
            *p++ = (unsigned char)value;
        }
        return dst;
    }
    
    /* You should implement your own kernel panic / logging. */
    __attribute__((noreturn))
    static void panic(const char *msg)
    {
        /* Replace with your own output method. */
        (void)msg;
        for (;;) {
            __asm__ volatile("hlt");
        }
    }
    
    /* Inline assembly helpers for x86 paging registers. */
    
    static inline void write_cr3(uintptr_t phys_addr)
    {
        __asm__ volatile("mov %0, %%cr3" :: "r"(phys_addr) : "memory");
    }
    
    static inline uintptr_t read_cr2(void)
    {
        uintptr_t val;
        __asm__ volatile("mov %%cr2, %0" : "=r"(val));
        return val;
    }
    
    static inline void enable_paging(void)
    {
        uint32_t cr0;
        __asm__ volatile("mov %%cr0, %0" : "=r"(cr0));
        cr0 |= (1U << 31);  /* Set PG bit */
        __asm__ volatile("mov %0, %%cr0" :: "r"(cr0));
    }
    
    /* Invalidate TLB entry for an address. */
    static inline void invlpg(uintptr_t addr)
    {
        __asm__ volatile("invlpg (%0)" :: "r"(addr) : "memory");
    }
    
    /* ================================
     *  Very simple physical frame allocator
     *  (bump-pointer, no free)
     * ================================ */
    
    /* Defined by your linker script: symbol at end of kernel image.
     * Example in linker script:
     *   ...
     *   _kernel_end = .;
     *   ...
     */
    extern char _kernel_end;
    
    static uintptr_t pmm_next_free_frame = 0;
    static uintptr_t pmm_max_frame       = 0;
    
    static void pmm_init(uint32_t memory_size_bytes)
    {
        /* Start allocating frames right after kernel end, page-aligned. */
        uintptr_t kernel_end = (uintptr_t)&_kernel_end;
        pmm_next_free_frame = (kernel_end + PAGE_SIZE - 1) & PAGE_MASK;
    
        /* Do not allocate beyond 'memory_size_bytes'. In a real kernel
         * you would use a memory map from the bootloader (e.g. Multiboot)
         * to avoid reserved/ACPI/BIOS areas, etc.
         */
        pmm_max_frame = (uintptr_t)memory_size_bytes & PAGE_MASK;
    
        if (pmm_next_free_frame >= pmm_max_frame) {
            panic("Not enough memory for frame allocator init");
        }
    }
    
    /* Very basic: allocate the next free 4KiB frame. No free implemented. */
    static uintptr_t pmm_alloc_frame(void)
    {
        if (pmm_next_free_frame >= pmm_max_frame) {
            panic("Out of physical memory");
        }
        uintptr_t frame = pmm_next_free_frame;
        pmm_next_free_frame += PAGE_SIZE;
        return frame;
    }
    
    /* Implement pmm_free_frame(frame) if you want to support freeing frames. */
    
    /* ================================
     *  Virtual memory manager internals
     * ================================ */
    
    static page_directory_t *kernel_directory = 0;
    static page_directory_t *current_directory = 0;
    
    /* Retrieve the page table for a virtual address.
     * If create != 0, allocate and initialize a new page table if missing.
     */
    static page_table_t *get_page_table(page_directory_t *dir,
                                        uintptr_t virt_addr,
                                        int create)
    {
        uint32_t pd_index = virt_addr >> 22;          /* bits 31..22 */
        uint32_t pt_entry = dir->entries[pd_index];
    
        if ((pt_entry & PAGE_PRESENT) == 0) {
            if (!create) {
                return 0;
            }
    
            /* Allocate a new page table. */
            uintptr_t new_table_phys = pmm_alloc_frame();
            page_table_t *new_table = (page_table_t *)new_table_phys; /* identity map assumption */
    
            k_memset(new_table, 0, sizeof(page_table_t));
    
            /* Mark PDE as present and writable (kernel mode). */
            dir->entries[pd_index] =
                (new_table_phys & PAGE_MASK) | PAGE_PRESENT | PAGE_WRITE;
            return new_table;
        } else {
            /* Extract physical address of existing page table. */
            uintptr_t table_phys = pt_entry & PAGE_MASK;
            page_table_t *table = (page_table_t *)table_phys; /* identity map assumption */
            return table;
        }
    }
    
    /* ================================
     *  Public VMM API
     * ================================ */
    
    void vmm_map(page_directory_t *dir,
                 uintptr_t virt_addr,
                 uintptr_t phys_addr,
                 uint32_t flags)
    {
        /* We always set PRESENT internally. */
        flags |= PAGE_PRESENT;
    
        page_table_t *table = get_page_table(dir, virt_addr, 1);
        uint32_t pt_index = (virt_addr >> 12) & 0x3FF; /* bits 21..12 */
    
        table->entries[pt_index] = (phys_addr & PAGE_MASK) | flags;
    
        /* If this is the current directory, flush the TLB entry. */
        if (dir == current_directory) {
            invlpg(virt_addr);
        }
    }
    
    void vmm_unmap(page_directory_t *dir, uintptr_t virt_addr)
    {
        page_table_t *table = get_page_table(dir, virt_addr, 0);
        if (!table) {
            return;
        }
    
        uint32_t pt_index = (virt_addr >> 12) & 0x3FF;
        table->entries[pt_index] = 0;
    
        if (dir == current_directory) {
            invlpg(virt_addr);
        }
    
        /* Optionally: if the entire page table is now empty, you could:
         *  - mark the PDE not-present
         *  - free the page table frame via pmm_free_frame()
         * This simple implementation skips that.
         */
    }
    
    uintptr_t vmm_virt_to_phys(page_directory_t *dir, uintptr_t virt_addr)
    {
        uint32_t pd_index = virt_addr >> 22;
        uint32_t pt_index = (virt_addr >> 12) & 0x3FF;
    
        page_directory_entry_t pde = dir->entries[pd_index];
        if ((pde & PAGE_PRESENT) == 0) {
            return 0;
        }
    
        page_table_t *table = (page_table_t *)(pde & PAGE_MASK); /* identity map assumption */
    
        page_table_entry_t pte = table->entries[pt_index];
        if ((pte & PAGE_PRESENT) == 0) {
            return 0;
        }
    
        uintptr_t phys_base = pte & PAGE_MASK;
        uintptr_t offset    = virt_addr & ~PAGE_MASK;
        return phys_base + offset;
    }
    
    void vmm_switch_directory(page_directory_t *dir)
    {
        current_directory = dir;
        /* Because kernel uses identity mapping early on, 'dir' is both a virtual
         * and physical address here. If you move page directories elsewhere
         * later, ensure you pass the physical address to CR3.
         */
        write_cr3((uintptr_t)dir);
    }
    
    page_directory_t *vmm_get_current_directory(void)
    {
        return current_directory;
    }
    
    /* Identity-map [start, end) in given directory. */
    static void identity_map_range(page_directory_t *dir,
                                   uintptr_t start,
                                   uintptr_t end,
                                   uint32_t flags)
    {
        uintptr_t addr;
        for (addr = start; addr < end; addr += PAGE_SIZE) {
            vmm_map(dir, addr, addr, flags);
        }
    }
    
    void vmm_init(uint32_t memory_size_bytes)
    {
        /* Initialize simple physical memory manager. */
        pmm_init(memory_size_bytes);
    
        /* Allocate and clear a page directory for the kernel. */
        uintptr_t pd_phys = pmm_alloc_frame();
        kernel_directory = (page_directory_t *)pd_phys; /* identity map assumption */
        k_memset(kernel_directory, 0, sizeof(page_directory_t));
    
        /* Identity-map some initial region of memory.
         *
         * You can choose:
         *   - only map kernel + data + stack + I/O regions
         *   - or map up to a fixed limit (e.g. 64 MiB) for simplicity.
         *
         * Here we map up to a fixed limit (64 MiB or memory_size_bytes,
         * whichever is smaller).
         */
        uintptr_t map_limit = memory_size_bytes;
        const uintptr_t MAX_IDENTITY_MAP = 64U * 1024U * 1024U; /* 64 MiB */
        if (map_limit > MAX_IDENTITY_MAP) {
            map_limit = MAX_IDENTITY_MAP;
        }
    
        identity_map_range(kernel_directory,
                           0,
                           map_limit,
                           PAGE_WRITE);  /* kernel pages, supervisor RW */
    
        /* Switch to kernel directory and enable paging. */
        vmm_switch_directory(kernel_directory);
        enable_paging();
    }
    
    /* Basic page fault handler example.
     *
     * error_code bits (for x86):
     *   bit 0: 0 = not-present, 1 = protection violation
     *   bit 1: 0 = read,          1 = write
     *   bit 2: 0 = supervisor,    1 = user-mode
     *   bit 3: 1 = reserved bit violation
     *   bit 4: 1 = instruction fetch (if supported)
     */
    void vmm_handle_page_fault(uint32_t faulting_address,
                               uint32_t error_code)
    {
        (void)faulting_address;
        (void)error_code;
    
        /* In a real kernel you would:
         *  - log the fault (virt address from CR2, error_code),
         *  - possibly grow stacks or handle copy-on-write, etc.
         * For now, just panic.
         */
        panic("Page fault");
    }
    

    3. Integrating into your kernel

    Linker script

    You need to export _kernel_end from your linker script so the allocator knows where free memory starts. Example:

    SECTIONS
    {
        /* ... your other sections ... */
    
        .bss ALIGN(4K) :
        {
            *(.bss*)
            *(COMMON)
        }
    
        _kernel_end = .;
    }
    

    Kernel entry example

    #include "vmm.h"
    
    /* Suppose your bootloader passes total memory size in bytes. */
    void kernel_main(uint32_t memory_size_bytes)
    {
        vmm_init(memory_size_bytes);
    
        /* Now paging is on and you can use vmm_map()/vmm_unmap(), etc. */
    
        /* Example: allocate one physical frame and map it at 0x400000. */
        /* For that you’d need to expose pmm_alloc_frame() or wrap it. */
    }
    

    Page fault ISR

    Register an ISR for interrupt 14 (x86 page fault). In the ISR, read CR2 for the faulting address and pass it plus the error code to vmm_handle_page_fault():

    extern void vmm_handle_page_fault(uint32_t faulting_address,
                                      uint32_t error_code);
    
    void isr_page_fault(uint32_t error_code)
    {
        uint32_t fault_addr = (uint32_t)read_cr2(); /* or inline CR2 read here */
        vmm_handle_page_fault(fault_addr, error_code);
    }
    

    (You can move the read_cr2() helper into a shared header if needed.)


    This gives you a working, minimal virtual memory manager:

    • Paging enabled and kernel identity‑mapped.
    • Page directory and tables managed in C.
    • Basic API for mapping, unmapping, and translating addresses.

    From here you can extend it with:

    • A real physical memory manager (bitmap/buddy using a BIOS/UEFI memory map).
    • Kernel heap (using a range of virtual addresses + vmm_map/vmm_unmap).
    • User‑space address spaces (one page_directory_t per process).
    • Demand paging / copy‑on‑write in the page fault handler.

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.