Develop a kernel module in C that adds a new system call to Linux.
- User
Develop a kernel module in C that adds a new system call to Linux.
Assistant
Developing the System Call
I'm focusing now on understanding the project's goal - a C kernel module adding a system call. I'm deep diving into how to do this, particularly exploring the system call table modification method. I'm weighing up the pros and cons of this approach compared to more modern alternatives and the implications for portability and maintainability.
Assessing System Call Methods
I've been evaluating different system call implementation approaches. While older tutorials heavily emphasized direct
sys_call_tablemanipulation and disabling write protection, this is now strongly discouraged due to security concerns. Modern kernels favor modifying the kernel source and recompiling or utilizing ftrace/kprobes for function hooking. For this task, I'm leaning toward the "syscall table hacking" approach, but I need to include significant disclaimers due to its inherent risks and unreliability. I'm focusing on acknowledgingkallsyms_lookup_nameand its potential implications.Outlining Code Implementation
Currently, I'm working through the code step-by-step. I've decided to demonstrate the "hacking" method, emphasizing its academic context. I'm focusing on hijacking
sys_mkdiras a target, as this avoids the complexities of defining a new system call number. The prerequisites are clear, and I'm outlining how to locatesys_call_table, save the original function, disable write protection, modify the table, and restore everything upon module exit. I'm taking notes on potential pitfalls.Adding a System Call
I'm now determining how to handle adding a new system call in the code. Hijacking
__NR_mkdiris tempting, but the task specifies an addition. This necessitates choosing an unused slot in the system call table. Locating this unused slot is the next challenge. For newer kernels (post-5.7),kallsyms_lookup_nameis no longer directly available, so I'll need akprobeworkaround. I'm focusing on crafting a helper function usingkprobefor symbol resolution.Finalizing System Call Logic
I'm now implementing the new system call function and working through the
kprobeworkaround for symbol resolution, sincekallsyms_lookup_nameisn't directly available in newer kernels. I've designed themy_new_syscallfunction and includedprintkfor verification. I'm focusing on the initialization and exit routines, and I've decided to hijack__NR_mkdirsince the user wants addition, I'll use it to showcase. I'm taking care to include appropriate safety mechanisms and disclaimers.Choosing a System Call
I've decided to proceed with implementing a replacement mechanism, showcasing how to modify a specific existing system call. I will use a
#definefor the chosen system call number, making it easily configurable. I'll make the replacement clear for the user to understand. I'm focusing on locating an appropriate, unused syscall.Preparing Code for User
I'm now integrating the code for presentation, including the Makefile, user application, and thorough instructions for compilation, loading, testing, and unloading the module. I'm focusing on making it as user-friendly as possible, with clear explanations of each step. I've re-examined the code, confirming syntactic correctness and the logic's soundness. The warnings about the approach's inherent risks are in place.
This is a complex task because the Linux kernel developers have intentionally made this difficult to improve security.
Historically, you could simply export the
sys_call_tablesymbol and modify it. In modern kernels (v5.7+), the kernel no longer exportssys_call_tableor evenkallsyms_lookup_nameto modules to prevent rootkits. Furthermore, the system call table is in read-only memory.To achieve this in a modern kernel (Standard Ubuntu/Debian/Fedora distributions today), we must:
- Use
kprobesto dynamically find the memory address ofkallsyms_lookup_name. - Use that to find the address of
sys_call_table. - Manipulate the CPU Control Register (CR0) to disable write protection temporarily.
- Hijack an existing (or unimplemented) system call vector.
⚠️ Critical Disclaimer
Do not run this on a production server. Modifying the system call table at runtime is unsafe. It can lead to system instability, crashes (Kernel Panics), and security vulnerabilities. This is strictly for educational purposes.
1. Validating the Kernel Version
This code is designed for x86_64 kernels.
2. The Kernel Module Source (
syscall_module.c)We will overwrite syscall 335. In many standard Linux kernels, vectors in the high 300s/400s are often unimplemented (pointing to
sys_ni_syscall), or we can hijack a defined one. For this example, we will define a new function and hook it into the table.#include <linux/module.h> #include <linux/kernel.h> #include <linux/init.h> #include <linux/unistd.h> #include <linux/syscalls.h> #include <linux/kprobes.h> MODULE_LICENSE("GPL"); MODULE_AUTHOR("Educational User"); MODULE_DESCRIPTION("A module to add a custom system call"); MODULE_VERSION("0.1"); // The system call number we want to hijack/implement. // 335 is usually free on many x86_64 kernels, but check /usr/include/asm/unistd_64.h or equivalent. // If 335 is taken, this will corrupt that syscall. #define NEW_SYSCALL_NUM 335 // Pointer to the original system call table unsigned long *sys_call_table_ptr; // Function pointer to store the original system call implementation (so we can restore it later) asmlinkage long (*original_syscall)(const struct pt_regs *); // Helper to resolve access to kallsyms_lookup_name (unexported in kernel > 5.7) typedef unsigned long (*kallsyms_lookup_name_t)(const char *name); kallsyms_lookup_name_t kallsyms_lookup_name_func; // Kprobe struct to find kallsyms_lookup_name's address struct kprobe kp = { .symbol_name = "kallsyms_lookup_name", }; // -------------------------------------------------------------------------------- // The Custom System Call Function // -------------------------------------------------------------------------------- // Since Kernel 4.17+, syscalls take a pt_regs struct on x86_64 asmlinkage long sys_my_custom_call(const struct pt_regs *regs) { printk(KERN_INFO "kernel_module: Custom system call %d invoked!\n", NEW_SYSCALL_NUM); return 0; // Return 0 for success } // -------------------------------------------------------------------------------- // Helper functions to bypass Write Protection (CR0 Register) // -------------------------------------------------------------------------------- static inline void write_cr0_forced(unsigned long val) { unsigned long __force_order; // Assembly to write to CR0 asm volatile("mov %0, %%cr0" : : "r"(val), "m"(__force_order)); } static inline void unprotect_memory(void) { // Bit 16 is the Write Protect (WP) flag write_cr0_forced(read_cr0() & (~0x10000)); printk(KERN_INFO "kernel_module: Write protection disabled.\n"); } static inline void protect_memory(void) { // Re-enable Bit 16 write_cr0_forced(read_cr0() | 0x10000); printk(KERN_INFO "kernel_module: Write protection enabled.\n"); } // -------------------------------------------------------------------------------- // Module Initialization // -------------------------------------------------------------------------------- static int __init mod_init(void) { printk(KERN_INFO "kernel_module: Loading...\n"); // 1. Find kallsyms_lookup_name using kprobes (the workaround for modern kernels) int ret = register_kprobe(&kp); if (ret < 0) { printk(KERN_ERR "kernel_module: Failed to register kprobe. Can't resolve symbols.\n"); return ret; } kallsyms_lookup_name_func = (kallsyms_lookup_name_t) kp.addr; unregister_kprobe(&kp); // We found the address, we don't need the probe anymore if (!kallsyms_lookup_name_func) { printk(KERN_ERR "kernel_module: kallsyms_lookup_name not found.\n"); return -EFAULT; } // 2. Find the system call table address sys_call_table_ptr = (unsigned long *)kallsyms_lookup_name_func("sys_call_table"); if (!sys_call_table_ptr) { printk(KERN_ERR "kernel_module: sys_call_table not found.\n"); return -EFAULT; } printk(KERN_INFO "kernel_module: sys_call_table found at %p\n", sys_call_table_ptr); // 3. Save the original syscall so we can restore it on exit // Note: We cast the address in the table to a function pointer original_syscall = (void *)sys_call_table_ptr[NEW_SYSCALL_NUM]; // 4. Overwrite the system call table unprotect_memory(); // Disable Read-Only sys_call_table_ptr[NEW_SYSCALL_NUM] = (unsigned long)sys_my_custom_call; protect_memory(); // Re-enable limit printk(KERN_INFO "kernel_module: Syscall %d hijacked.\n", NEW_SYSCALL_NUM); return 0; } // -------------------------------------------------------------------------------- // Module Cleanup // -------------------------------------------------------------------------------- static void __exit mod_exit(void) { if (sys_call_table_ptr) { unprotect_memory(); // Restore the original system call sys_call_table_ptr[NEW_SYSCALL_NUM] = (unsigned long)original_syscall; protect_memory(); printk(KERN_INFO "kernel_module: Syscall %d restored.\n", NEW_SYSCALL_NUM); } printk(KERN_INFO "kernel_module: Unloaded.\n"); } module_init(mod_init); module_exit(mod_exit);
3. The Makefile
You need the kernel headers installed for your current kernel (
sudo apt install linux-headers-$(uname -r)on Debian/Ubuntu).Create a file named
Makefile:obj-m += syscall_module.o all: make -C /lib/modules/$(shell uname -r)/build M=$(PWD) modules clean: make -C /lib/modules/$(shell uname -r)/build M=$(PWD) clean
4. Userspace Tester (
test_syscall.c)Since this is a new/hijacked syscall,
glibcdoesn't know about it. We must use thesyscall()function fromunistd.h.#include <stdio.h> #include <unistd.h> #include <sys/syscall.h> // Must match the ID in the kernel module #define NEW_SYSCALL_NUM 335 int main() { printf("Invoking kernel syscall %d...\n", NEW_SYSCALL_NUM); // Invoke the syscall // Arguments: Syscall Number, optional args... long res = syscall(NEW_SYSCALL_NUM); if(res == 0) { printf("System call returned success!\n"); printf("Check 'dmesg' to see the kernel log output.\n"); } else { perror("System call failed"); } return 0; }
5. Instructions to Run
-
Compile the Module:
makeThis generates
syscall_module.ko. -
Compile the Tester:
gcc test_syscall.c -o test_syscall -
Load the Module:
sudo insmod syscall_module.ko -
Check Kernel Logs:
dmesg | tailYou should see "Loading...", "sys_call_table found...", and "Syscall hijacked".
-
Run the Tester:
./test_syscallIt should output "System call returned success!".
-
Verify Execution: Run
dmesg | tailagain. You should see:kernel_module: Custom system call 335 invoked! -
Unload and Clean Up:
sudo rmmod syscall_moduleNote: Always unload the module before shutting down or modifying the code, as leaving the syscall table pointing to unallocated memory (after the module is removed) will instantly crash the OS next time something calls that ID. The code provided handles restoration in the
mod_exitfunction.
- Use
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.